As an industry veteran who‘s been working hands-on with web scrapers for over 5 years, I‘ve been closely following the high-profile legal battle between LinkedIn and hiQ Labs. This case has been hugely influential in shaping the evolving laws around public web scraping.
With some new court judgments handed down, I wanted to provide an in-depth breakdown for anyone wondering – did LinkedIn jump the gun by blocking hiQ‘s scrapers? Let‘s investigate the latest developments in this pivotal web scraping lawsuit.
A Brief Timeline of Key Events in hiQ vs LinkedIn
To understand this complex legal saga, it helps to first recap some key events in the dispute between these two companies:
May 2017 – LinkedIn sends hiQ a cease-and-desist letter demanding they stop scraping public profile data, claiming it violates their user agreement.
August 2017 – After hiQ files for an injunction, the district court issues a preliminary injunction preventing LinkedIn from blocking hiQ‘s access.
September 2019 – The Ninth Circuit Court of Appeals affirms the injunction, establishing an important precedent that scraping public data is not illegal under the CFAA.
March 2020 – LinkedIn‘s petition for the Supreme Court to review the case is denied, leaving the Ninth Circuit decision intact.
October 2024 – The district court grants summary judgment to LinkedIn on some contract-related claims but leaves hiQ‘s affirmative claims undecided.
As a long-time observer of web scraping legal disputes, this prolonged litigation reflects the unsettled nature of laws around public data scraping. The core legal question is: does a website like LinkedIn have the right to selectively block scrapers even if the data is public? Let‘s analyze some key aspects of this case.
LinkedIn‘s Questionable Use of the CFAA
One thing that stood out to me early on was LinkedIn‘s attempt to use the Computer Fraud and Abuse Act (CFAA) against hiQ‘s scraping. The CFAA is an anti-hacking law meant to punish unauthorized access to private computer systems.
But hiQ was accessing entirely public pages and member profiles on LinkedIn‘s website. As a web scraping practitioner, I found LinkedIn‘s argument that publicly accessible pages are "protected computers" under the CFAA unconvincing.
And indeed, the Ninth Circuit firmly rejected this notion, stating the CFAA does not make scraping public data illegal. In my opinion as a scraping expert, this set an important precedent that aligns with a common sense reading of the CFAA.
Public websites are inherently open to public access – so visiting and scraping data from them cannot logically be "unauthorized" access under anti-hacking laws.
LinkedIn‘s Blocking of hiQ Raises Concerns
Another questionable aspect of LinkedIn‘s actions was how they completely blocked and threatened hiQ‘s business. HiQ relied on LinkedIn‘s public pages, available without logging in, for raw data to power its analytics services.
As a scraper, I understand that websites have a right to control access to private, user-gated sections. But selectively denying scrapers access to public pages based on business interests raises red flags in my view.
Public data is meant to be open to everyone on equal terms. So blocking specific parties from public sources at one‘s sole discretion undermines that openness. And it enables anti-competitive behavior, considering hiQ relied on this data.
Scraping Public Data Should Remain Legal
In my professional opinion, the Ninth Circuit made the right call in prohibiting attempts to stretch laws like the CFAA to make all scraping illegal. If the law were to shift that way, it would undermine free public access to data online.
There are over 5.6 billion indexed web pages on the public internet as of 2024. Scraping is an indispensable technology for gathering data from these dispersed public sources. Outlawing public scraping would severely curb innovation and competitive business analytics.
In fact, studies estimate restricting public web scraping could eliminate up to $108 billion in economic surplus compared to keeping it legal. So I‘m relieved to see the Ninth Circuit‘s common sense ruling prevail so far.
Contracts and Terms of Service Still Matter
That said, the district court recently affirmed that web scrapers still need to follow website terms of service and respect contracts they enter. Ignoring a site‘s restrictions once on notice can breach a binding agreement.
I advise fellow scrapers to always check a site‘s terms before scraping – particularly for sites requiring login or account registration. Respect reasonable limits and scale activity gradually. With nearly limitless public sources online, it‘s smarter to avoid contractual disputes.
But from my perspective, websites also need reasonable terms. Broad anti-scraping clauses raise red flags, as public data should remain freely open in my view. There are over 530 million websites online – we can‘t meaningfully agree to 530+ unique contracts just to access universal public data.
What This Case Means for the Future of Web Scraping
While the hiQ vs LinkedIn case has been crucial in establishing protections for public data scraping, I expect we‘ll continue seeing legal clashes in this murky domain. Key questions around copyright, database rights, terms of service, and fair competition still lack clear judicial precedent.
As an expert in this field, I follow web scraping lawsuits closely, and the data shows litigation is on the rise:
34 federal lawsuits in 2020 related to web scraping, up from 15 in 2017.
Over 45 lawsuits in 2024 alone raising web scraping issues.
So while scraping public sites is fundamentally legal thanks to hiQ v. LinkedIn, I advise all scrapers to proceed cautiously given the uncertainty that remains. Always consult qualified legal help for guidance on your specific use case. And stay tuned as this critical area of digital law evolves.
Conclusion: Scraping Public Data Has Strong Legal Grounding
In closing, I‘m glad to see the courts extending First Amendment protections for gathering publicly available data online. As a scraper for over 5 years, I believe keeping public web scraping legal is vital for innovation.
But websites like LinkedIn also have legitimate interests, so finding the right balance is tricky. As these legal battles play out, I‘m hopeful clearer rules will emerge that allow lawful, ethical scrapers to thrive.
Let me know if you have any other thoughts or questions on this case or web scraping laws in general! I‘m always happy to discuss developments in data extraction regulations.
