Data pipeline and employment services platform for BC job seekers
50K+
Job listings indexed
WCAG 2.1 AA
Accessibility standard
100+
Data sources scraped
99.9%
Uptime SLA
WorkBC — workbc.ca
The challenge
The BC Government needed a reliable, scalable system to aggregate job postings from hundreds of employers across British Columbia into a single authoritative source for job seekers. Existing processes were manual, inconsistent, and couldn't scale to cover the volume of listings needed to serve the province effectively. The platform also needed to meet WCAG 2.1 AA accessibility standards — a non-negotiable requirement for any BC Government digital property — while handling the performance expectations of a public-facing portal used by thousands daily.
What we built
We designed and built a multi-source job data pipeline using Python and Scrapy to crawl and normalize listings from employers, job boards, and government sources province-wide. The pipeline runs on Azure with automated scheduling, deduplication, enrichment, and indexing into PostgreSQL — feeding a filterable public job board built in React. We implemented full WCAG 2.1 AA compliance across the UI, integrated with Azure Active Directory for internal tooling, and set up automated monitoring and alerting so the pipeline stays healthy without manual babysitting. The result is a production system trusted by the BC Government to power employment services for hundreds of thousands of residents.
About the project
WorkBC is the provincial employment services program supporting job seekers and employment service providers across British Columbia. We built and maintain their job data pipeline — scraping, normalizing, and indexing tens of thousands of job listings from across BC into a searchable, filterable platform. We also delivered custom platform features and a WCAG-compliant public-facing web presence backed by government-grade Azure infrastructure.
What we delivered
- ✓Job data pipeline — scraping, normalizing, and indexing listings province-wide
- ✓Province-wide deployment serving hundreds of thousands of job seekers
- ✓WCAG 2.1 AA accessible public-facing interface
- ✓Azure Government cloud infrastructure with automated monitoring
- ✓BC Government — long-term client relationship
Tech stack
How we worked
01
Discovery & Architecture
Mapped all job data sources across BC, defined ingestion, normalization, and deduplication rules, and designed a pipeline architecture for Azure that met government compliance requirements.
02
Pipeline Development
Built Scrapy-based crawlers for each data source, a normalization layer to harmonize job fields across sources, and a PostgreSQL schema optimized for full-text search and faceted filtering.
03
Platform & UI
Built the React frontend with server-side rendering for SEO, implemented WCAG 2.1 AA compliance across all components, and integrated with the pipeline via a Node.js API layer.
04
Infrastructure & Launch
Deployed on Azure with autoscaling, set up CI/CD via GitHub Actions, wired up monitoring and alerting, and handed off runbooks to the WorkBC team before go-live.
Outcome
Province-wide deployment supporting hundreds of thousands of BC job seekers
government web development Vancouver · BC government software development · employment services platform · job board development British Columbia · data pipeline development BC · WCAG compliance web development · Azure government cloud Canada · government software agency Vancouver · Python scraping pipeline · public sector web development · accessibility web development BC · React web application government · web development agency British Columbia · software development Vancouver BC · custom web platform BC government
Ready to build?
Tell us about your project. If it's a good fit, we'll build it — and add it here.
Start a Project