Clifford Chance

Why do you sometimes have to click those diagrams before you can log into a social media platform? Because the website wants to know you are a human – and not a bot looking to scratch its surface with a view to obtaining valuable data (a "data scraper"). Data scrapers extract, aggregate and combine data from various sources. Scraped data is then generally stored on a local system and used for different purposes such as recruitment, sentiment analysis, assessing credit risk, identifying trends, marketing and sales. 

What is data scraping?

Scraped data can be stored, copied or analysed for various new purposes. Use cases for data scraping include:

• contact scraping – e.g. retrieving email addresses of businesses / individuals to compile a mailing list
• web scraping – e.g. accessing the underlying code of a website and copying data from that code
• competitor monitoring – e.g. retrieving price information of competitors, including for monitoring purposes / to help advance an organisation's competitive edge
• reputation monitoring – e.g. scraping comments made on social media platforms or review sites to monitor business reputation
• screen scraping – e.g. using scraping tools to emulate a human end-user to extract publicly available data on a large scale

Scraping is often invisible. The scraper knows what they are doing – the scraped site may not. The harvesting will often take place contrary to the terms of use and privacy statements of the sources that are scraped. Regulators want to limit this exposure.

Who is at risk and from what?

Depending on the jurisdiction, data scrapers may find themselves on the receiving end of legal action by a scraped business under various regimes, including: (i) contract; (ii) intellectual property; (iii) computer misuse/fraud rules; and (iv) data protection and privacy.

What are regulators saying?

Global awareness and intervention on data scraping is increasing globally, including in the privacy context. On 24 August 2023, a joint statement on data scraping was issued by a club of global privacy regulators from Australia, Canada, the UK, Hong Kong, Switzerland, Norway, New Zealand, Columbia, Jersey, Morocco, Argentina and Mexico.  The statement focusses specifically on privacy considerations, in light of data protection authorities seeing "increasing incidents involving data scraping, particularly from social media and other websites that host publicly accessible data". 

The joint statement emphasises that scraped data can be "exploited" for various purposes, such as monetization through re-use on third party websites, sale to malicious actors, or private analysis or intelligence gathering, resulting in serious risks to individuals, encouraging relevant organisations to protect against unlawful data scraping.

The joint statement also provides specific privacy recommendations, such as: (i) monitoring (e.g. how quickly a new account starts looking for users); (ii) taking steps to identify 'bot' activity (e.g. monitoring suspicious IP addresses and using CAPTCHAs); and (iii) considering legal avenues of redress (see below).

What to do?

In our previous article Scraping the barrel?Legal issues arising from data scraping, we make six practical recommendations for reducing the risks for organisations whose activities are impacted by data scraping:

• Know the law: Various local laws already regulate whether (and how) organisations can use scraping technology to analyse data. The GDPR, for example, may need to be carefully considered to ascertain whether the proposed scraping activity is permitted by law at all.
• Update website terms and privacy notices: Organisations should carefully check whether their website terms restrict use and access to content uploaded to their sites – e.g. work product, or personal information. The enforceability of these terms will vary across jurisdictions, however, at the very least, robust contractual provisions help set expectation and intention. Privacy notices should be clear about any proposed data sharing and scraping activities.
• Know your supply chain:  Regulators expect organisations using third party analytics companies to ensure that they carry out sufficient checks and diligence their supply chain. The leaders will build practical checks to audit their suppliers' compliance with data rules.
• Technical safeguards: Website owners should check that they take appropriate measures to safeguard personal information they process, taking into account what technology is available in the market to protect against the risk, particularly to the extent that their processing is subject to EU data protection rules. 
• Soft regulation: Regulators globally are increasingly emphasising that complying with the law is not enough. Scraping may be legal in some cases – but is it ethical? Not asking the right questions creates up risk from a legal and reputational perspective. Asking questions about why and how data scraping is used is critical.
• Continuing audit: The GDPR requires organisations to mandatorily carry out a detailed data protection impact assessment where their processing of personal data poses a high risk to individuals.