Google News
logo
Full Stack Developer - Interview Questions
Do you know how to prevent a bot from scraping your publicly accessible API?
As long as the data within the API is accessible to the public, it will technically not be possible to completely prevent data scraping. It is possible, however, to minimize bot activity (automated computer programs on the internet that perform certain tasks) by throttling or rare limiting. Rare limiting will be able to prevent a certain device from making an unlimited number of requests within a defined time.

If too many requests are made beyond the defined limit, a 429 Too Many Attempts HTTP error is thrown. It is vital to record more than just the IP address of the device since the IP address is not unique to each device and can stop the whole network from accessing the API.
Advertisement