The University of Michigan is one of the top public universities in the United States, ranked #1 by US News and World Report with over 6 million books in their library. With prestigious programs in data science and information technology, they are shaping tomorrow‘s leaders and innovators.
To provide their students with expertise in the rapidly growing field of web scraping, the University of Michigan collaborated with Oxylabs – a leading provider of web data extraction tools and services.
Widespread Adoption Driving Need for Education
Web scraping now plays a critical role in public and private sector industries:
- 82% of companies use web scraped data according to recent surveys.
- The web scraping market size is expected to grow to $13.9 billion by 2027.
- 90% of data scientists report using web scraping in their daily work.
However, there is still widespread uncertainty regarding the legality and ethics of proper web scraping techniques. This partnership aimed to directly address that knowledge gap through educational offerings specially designed for University of Michigan‘s graduate curriculum.
Sharing Insights into the Legal Landscape
Oxylabs provided two educational sessions for students in the University‘s Master of Applied Data Science program:
1. Panel Discussion on Web Scraping Laws
Moderated by Oxylabs‘ Head of Legal Denas Grybauskas, this virtual panel covered topics ranging from copyright law, data privacy, and Terms of Service violations. Students gained broad exposure to the complex legal landscape around web scraping.
2. Lecture on Ethical and Legal Considerations
This detailed lecture dove deeper into key issues like data ownership, personal data sensitivity, GDPR, relevant legal cases, and Oxylabs‘ risk assessment model.
"It allowed us to root the discussion of what actions are appropriate in the legal precedents formed over the last twenty years." – Christopher Brooks, University of Michigan
Oxylabs‘ insights, drawn from real-world experience, are invaluable for students exploring this emerging field.
Gaining Hands-On Skills
In addition to lectures, Oxylabs hosted an intensive workshop where students used the company‘s tools like proxies and browsers to scrape public web data for educational purposes.
Specific activities included:
- Testing different locate/identify techniques for scraping dynamic content.
- Experimenting with proxies to manage requests and IP rotations.
- Using headless browsers to render JavaScript pages.
- Implementing delays and other practices to avoid overloading servers.
This hands-on learning was a huge benefit, allowing students to put legal theory into ethical practice.
Why Oxylabs Values Educational Partnerships
For Oxylabs, collaborations with academia provide an opportunity to nurture new talent and shape the industry‘s future. As experts in the field, they understand the pressing need for formal web scraping education.
Trends they have observed include:
- Rising demand for people with web scraping skills.
- New job roles like Data Compliance Officer emerging.
- A lack of standardized curriculums on proper techniques.
That‘s why Oxylabs gladly provides resources to universities like Michigan‘s School of Information. As Julius Černiauskas, Oxylabs‘ CEO stated:
"It‘s significant for industry leaders to give a hands-on learning experience for students who will soon potentially join our industry."
More partnerships between industry and academia will continue advancing web scraping best practices – encouraging compliance and innovation from the ground up.
The Road Ahead
Moving forward, Oxylabs plans to build on this initial collaboration to further enrich Michigan‘s curriculum via more hands-on learning workshops, technical insights, and resources.
They hope this partnership serves as a model for how academia and industry can work together to develop ethical web scraping experts. As fields like data science continue evolving rapidly, it is crucial we provide students with a strong foundation.
I‘m encouraged to see companies taking responsibility and partnering with universities. Formal education will enable us to maximize the value of web scraping, while minimizing legal and reputational risks.
