While working on a Python monitoring project, I encountered an issue where searxNG stopped functioning effectively. Though technically operational, the meta search engine was being blocked by services like Google and Bing, which were throwing CAPTCHAs or outright blocking my requests.
As an alternative, I switched to Google Custom Search Engines (CSE) – also known as Programmable Search Engines (PSE). These are powerful tools that allow you to create predefined search parameters that remain constant while only changing specific variables like names, phone numbers, or email addresses.
What makes CSEs useful:
- You can create a search with multiple fixed criteria and share the URL with colleagues
- They can be configured to search specific sites with certain restrictions
- Most importantly, they can be accessed via API, making them excellent for monitoring projects
Important pricing information:
While the CSE page indicates you can have up to 10,000 queries per day, there’s a crucial detail not prominently mentioned: only the first 100 queries per day are free. Beyond that limit, you need to:
- Create a billing account associated with your CSE project
- Provide payment information (credit card)
- Pay $5 USD per 1,000 queries after the free daily allowance
This pricing model is actually quite reasonable for what the service provides, and it ensures your monitoring won’t suddenly stop working when you exceed the free tier. The system will continue to function while charging you at the established rate.
If you’re considering alternatives to meta search engines like searxNG for monitoring projects, Google’s CSE/PSE option might be worth exploring – just be aware of the billing structure before you begin.
