Web scraping, also known as web harvesting and web data extraction. Essentially web scraping is to extract data from the websites and it is commonly used in the data collection stage. At DP Financial, to ensure the data can be automatically downloaded with greater accuracy, we can write a well-designed and customized scraping program for the clients.
1
Case Study: data from financial report
Database Involved: SEC EDGAR
​
Software: Python, MS Excel
​
Service Provided:
​
Our client needed to check the prospectus lodged by 1,620 mutual funds from 2006 to 2019. Instead of manually search for the mutual funds on SEC and click on their prospectus, we have written a program using Python to search the funds on SEC automatically and download the links to each of the prospectus within the sample period as per client’s requirement. To ensure the accuracy of the data collected, a checking mechanism is built into the program to verify the data extracted.