Data Engineer

To give you a sense of the gap: awarded quotes that slip through the platform get manually caught by our commercial team and hand-entered into HubSpot instead. Supplier onboarding data follows the same pattern. So as and when the platform and HubSpot become out of sync, it makes it difficult to say with confidence which one is right.

This is a build-and-do role. You'd collaborate closely with our Head of Engineering to shape the architecture, and you'd also write the pipelines, carry the pager for them, and go and read the raw source when a number looks wrong. If you're looking for a role where you set direction in isolation and hand implementation to other people, this isn't it. If you want substantial responsibility and scope as a function lead alongside our Head of Engineering while still enjoying the craft yourself, it very much is.

You would not be starting from nothing. There's a live platform with real transactions running through it, a product engineering team who know where the bodies are buried, and access most people in this field would have to scrape for: Civic Marketplace is a member of the NIGP Business Council, with real partnerships across councils of governments. Most people doing this job spend their first year getting the data access. You'd start with the door already open.

What you'd own

Five things, and the freedom to decide how.

• Trust in the data. The north star. A golden, trusted dataset that powers Civic Marketplace's analytics, so when someone asks how many quotes were awarded last quarter, or how much GMV we've captured, there's one number, and it's right. You'd own the diagnosis and the fix.
• Ingestion. Pipelines that pull supplier, agency, solicitation and contract data out of sources never designed to be read by anything but a person. Reliable, observable, and cheap enough to add the next source without a meeting about whether it's worth it.
• The canonical model. One supplier, one record, across every spelling, trading name, subsidiary and registry identifier. One shared definition of a contract vehicle, a commodity, an agency, that product, sales and customer success all use and none of them argue with. This is the unglamorous part that makes everything downstream possible.
• The system underneath. Testing, lineage and freshness, so when the platform tells an agency something we know where it came from and when. And self-serve data for product, customer success and analytics, so nobody queues behind an engineer for a number. Everything you build should work for the next source without you in the loop. This is where AI earns its place, turning what is currently hand-checked into something repeatable.
• Agentic data infrastructure. As we scale our use of AI-native procurement, you will own the data layer that powers our agentic workflows. This goes beyond standard retrieval. You will design ingestion and storage strategies, including optimizing vector pipelines like Pinecone for RAG, to ensure our agents have the high-fidelity, context-aware data necessary to perform reliably in production.

You'd work closely with product engineering, customer success and our Growth Lead. Application feature work stays with the product engineering team, so you're not competing for that ground. You own the layer they build on top of.

How we use AI

We use Claude daily, across content pipelines, community response and internal knowledge, and increasingly inside the product itself. That's real, not a line in a job ad.

For this role it cuts two ways. There's how you work: source profiling, schema mapping, test generation, the documentation that otherwise never gets written. Most of that is now automatable, and we expect you to automate it. And there's what you build: the data layer that decides whether agentic procurement is trustworthy or embarrassing. The second is the harder and more interesting problem.

What we care about is not whether you can use it. Everyone says they can. It's whether you reach for it to make something repeatable. The difference between someone who writes pipelines faster and someone who builds the tooling that means the next hundred sources don't each need hand-holding is the difference we're hiring for.

If you've built data quality tooling, evaluation harnesses, retrieval pipelines or automated documentation from scratch, tell us about them. If AI has changed how you think about the work and not just how fast you produce it, we especially want to hear that.

Our stack: Snowflake, Postgres, dbt, Fivetran, HubSpot, PostHog, Metabase.

What you bring

You'll need:

• Data engineering experience in a startup or scale-up, where you've built the platform rather than inherited one
• Strong SQL and Python, and the judgement to know when the warehouse is the wrong place to solve a problem
• A data model you designed that other people had to live with, including the parts you would do differently now
• Real experience of messy external sources: half-documented APIs, flat-file drops, scraped pages, PDFs, and data you neither control nor can correct
• Entity resolution or record linkage experience, or clear evidence you'd be good at it. This is the centre of the job, not an edge case
• A commercial head. You can tell the difference between a data problem that's blocking revenue and one that's merely interesting, and you sequence accordingly
• The instinct to find out why a number is wrong before proposing a fix. Sometimes it's the pipeline, sometimes the model, sometimes the source was always like that, and those need different answers
• The instinct to build systems that scale rather than pipelines that run once
• Genuine AI fluency, in the sense above
• The determination to stay positive through the ups and downs of a fast-moving startup, and to find a path through problems rather than wait for conditions to be right
• Comfort operating remotely across timezones with real autonomy and not much oversight
• Curiosity about why public sector data is different. You don't need to have done it. You do need to want to understand it

You'll stand out if you have:

• Govtech, public sector or civic tech background, or experience working with public records and open data
• Familiarity with supplier and procurement data: SAM.gov and UEI, DUNS, NA