Custom Data Pipeline Development
I build data pipelines: extract from APIs, databases or files, transform and clean the data, and load it where you need it — usually Elasticsearch, MySQL or S3. Built to handle volume and to be observable.
What I Can Help With
Build ETL/import/transform/export pipelines for large datasets.
Typical Problems
Relevant Experience
Built ETL pipelines for Cenoskop ingesting product data from 500+ sources, transforming and normalizing it, then loading into Elasticsearch and MySQL. Handles millions of records daily with schema validation and error recovery.
What Is Included
How It Works
We define sources/targets and SLA, build the pipeline with validation and monitoring, and run a first load. From €2,000.
Starting from €2,000
Pipeline for a primary source/target pair.
Frequently Asked Questions
How do you handle schema drift?
I version and validate schemas at the extraction boundary, and alert on drift rather than failing silently.
Related Services
Systems I've Built
Cenoskop / Levnobot
Crawler monitoring millions of products. Processing 30M products daily and comparing them.
Built and maintain the full crawling pipeline — from scraping through Elasticsearch indexing to price comparison.
Niftycent
Price intelligence engine for the US market. Processing large-scale product data.
Built the data pipeline and crawling infrastructure for large-scale US product monitoring.
AnalyticsForWebsite
Custom solution for collecting and evaluating traffic metrics with emphasis on processing speed and minimizing backend load.
Designed and built the backend for fast traffic metric collection and evaluation.
facha.sk
Find a job or offer your services.
Built the Go backend — API, matching logic and deployment.
Paidshield
Chat application with end-to-end encryption.
Built the backend and encryption layer for a secure chat application.
remotedaemon.com
Remote security scanner with ping and certificate checking.
Built the scanning daemon and monitoring backend.
Synopsee
A Cleaner, Safer Internet. Take control of your DNS. Block ads, trackers, and malicious sites with customizable profiles.
Built the DNS resolution backend and profile management system.
Not sure what's wrong?
Write what's happening and paste anything relevant — errors, logs, metrics. I'll have a look and tell you what I think.