Hey,

Actually working on a web scrapping project with a friend, and we would like to know public databases / apis to query.
Actually doing a good part of the predev phase on www.ncbi.nlm.nih.gov


Guys, do you have other public database / apis to be queried ?
Do you have worked on a such project ? and have some tips to share ?

Our needs are :

- Querying as on google, but specific APIs / DBs concerning plant, with their phyto, chemicals, scientific paper
- Gather the data, arrange them and store into local DB
- Download files / document store it

We choose to use python with scrapy for querying and collecting data, then use an ELK stack for viewing / analysis.
This may evolve in the futur, but we found the idea interesting, as it could be adapted to other themes. If something is produced and work a bit, the code will be posted -as it is-, as it's starting.