Developer creates Python job aggregator combining HH, Habr Career, and GeekJob

A developer, tired of manually checking five or more job boards every morning, wrote his own job search aggregator in Python and published the source code openly on Habr.

How the aggregator works

The tool solves a typical pain point for job seekers in the technical field: relevant vacancies are scattered across a dozen different sources — from major aggregators like HH.ru to niche platforms like Habr Career and GeekJob, as well as numerous thematic Telegram channels where recruiters and HR teams publish offers directly. Daily manual checking of all these sources takes time, and a significant portion of vacancies are duplicated across multiple platforms simultaneously.

• Data sources — HH, Habr Career, GeekJob and Telegram channels
• Implementation — Python script
• Key function — automatic removal of duplicate listings
• Delivery of results — notifications in Telegram
• Project source code is open

Why such tools appear regularly

The idea of combining several job sources into one stream is not new: similar functionality has been offered for years by built-in subscriptions from the job boards themselves, as well as by third-party services and Telegram bots for the job market. But most platforms lack an official public API, or it is severely limited, so developers who need flexible filtering and deduplication logic for themselves often write their own scripts — relying on official APIs where they exist, and on parsing public pages where APIs are absent.

Publishing the source code openly is a typical practice for such personal tools on Habr: it allows other developers not to write from scratch the logic of integration with already known sources, but to refine the ready-made base for their own stack and set of platforms.

Why deduplication is the most non-trivial part

At first glance, combining multiple job sources looks like a simple task, but in practice, the most difficult thing in such aggregators is not data collection, but its subsequent normalization. The same vacancy is often posted by an employer simultaneously on several platforms with different titles, slightly different descriptions of duties or different formatting of the salary range, and recruitment agencies often duplicate the same position under different company names. To filter out such duplicates, developers need to compare not exact strings but semantic similarity of text — by keywords, company name and matching salary range — which already goes beyond simple parsing and requires more flexible matching logic.

Delivering the result via Telegram is also a deliberate choice: the messenger has become the de facto main channel for the Russian IT job market, where a significant portion of vacancies is published through thematic channels and bots, not just through classic job boards.

What this means

The story is a small but telling example of how job seekers and developers solve the problem of information fragmentation in the labor market themselves, without waiting for a unified aggregator from major players: as long as such a universal solution does not exist on the market, niche open source projects like this continue to fill the gap for individual users.