More than a decade ago, when I was working as a team lead and later as a project manager, I participated in countless hiring processes. Mostly the technical sections – the recruiters handled finding candidates, and I assessed whether they could actually do the job in our data engineering and business intelligence implementation projects.
And I kept noticing a pattern that frustrated me.
Really interesting people – talented, skillful, genuinely motivated to join our company – would come through the process with bright eyes and high energy. But then, six months after they started working on real projects with real systems, something shifted. They seemed a little unhappy. Sometimes they’d admit it directly: this wasn’t what they signed up for. Their reality check hit harder than they expected.
At first, I felt frustrated. “Didn’t you know where you were applying? Didn’t you research what you’d actually be doing?” I’ll admit I even felt some irony toward them – which, thankfully, evolved into empathy as I got more involved in onboarding. I started trying to help them adapt rather than judging them for not reading the fine print.
But the pattern kept repeating. And eventually, I had to ask myself: what’s actually going wrong with our approach to hiring data engineers?
The sales machine I didn’t know existed#
As I became more involved in hiring – recruiting for bigger projects, growing teams, expanding my area of responsibility – I finally saw what was happening behind the scenes.
My colleagues in HR weren’t just recruiting. They were selling vacancies.
Every candidate who seemed remotely suitable got the full pitch. The company brand, which was strong among both customers and employees, amplified everything. It wasn’t like people were lining up to work there the way they might for Google or Apple, but it was a factor. And the recruiters? They were doing exactly what we expected them to do.
When I talked to them about it, they explained their reality. They were competing for talent in a tough market. They wanted to win. They had their own KPIs, their own pressure to fill roles quickly and with the best people available. Of course they sold hard – that’s literally the job in AI talent acquisition.
And it worked. We hired talented people.
But then came those disappointed data engineers six months later. Our data team retention was suffering, and I couldn’t figure out why.
Same pattern, different company#
When I started my own company, I thought I understood the problem well enough to avoid it.
I was wrong.
After spending thousands of hours in B2B sales – selling myself, building our pipeline, speaking with prospects who didn’t even want to buy our services – I naturally applied the same approach to hiring senior data engineers. I pitched candidates on the opportunity. I gave them visibility into our roadmap, the perks, the growth potential. I did my best to persuade skilled people to join us.
And it worked. Until it didn’t.
Same pattern emerged. Employees who started strong but gradually lost that spark. A gap between what they expected and what their day-to-day reality actually looked like.
Thankfully, we have a culture of experimentation and fast iteration, so we started fixing it. And what actually worked came from an unexpected place – deeper in the sales playbook than I’d originally gone.
The missing piece from B2B sales#
Here’s what was missing from every recruitment process I’d been part of, including my own: we never worked with the candidate’s actual demand.
In B2B sales, you don’t just pitch your product and hope for the best. You try to understand what problem the prospect is actually trying to solve. What’s their intent to buy? What need are they hoping to address?
We weren’t doing any of that with candidates.
I don’t mean discovering some crisis in their life that they desperately need to fix. Not every B2B prospect is in a desperate situation either. What I mean is understanding their motives, their requests from their professional life, the things they’re hoping might be answered by joining your company.
And this goes far beyond money. Far beyond learning new technology or having technically challenging projects. We’re humans. We’re more than professional functions, even in the well-structured B2B world of data science and engineering services with all its management frameworks and established practices.
We bring our self-image to work. Our expectations about relationships and connections. How we want to be perceived socially – and let’s be honest, for most of us, our social life is largely our professional life. We have needs for recognition, for expressing creativity even in technical roles. All of these things can form a motive for changing jobs, changing companies, even changing technical profiles entirely.
Understanding this became the foundation of our cultural fit methodology – and it changed everything about how we approach data team building.
The shift that changed our retention#
So we evolved our approach.
Instead of immediately assessing technical skills and selling the role, we started working with candidates’ actual needs. We tried to discover them, understand them, interpret them correctly.
If we saw that we could genuinely solve their problem – that our company could be an answer to what they were actually looking for – we proceeded to a deeper technical assessment and the rest of the hiring process.
If not, we communicated that honestly and let them go. Even if they were a technical genius who theoretically fit our team perfectly.
This is fundamentally different from most hiring frameworks that I’ve been aware of. It’s also different from chasing that mysterious “highly motivated candidate” who seems willing to do anything to get the position. Yes, that person might be motivated – but motivation fueled by a fantasy version of your company, or by completely wrong expectations about the actual work, doesn’t help anyone.
We shifted from “choosing the ones who want to work with us” to “working with those we can actually bring value to.” Not just in material terms like salary, and not just in terms of professional development.
Because here’s the thing – in sales, the most grateful, successful, long-term clients are usually the ones whose needs were genuinely well-matched to your product. The most painful clients are the ones who got pushed through the sales process and persuaded that your product was what they needed when it really wasn’t.
Same principle applies to building data engineering teams.
The efficiency question#
At this point, you might be thinking: understanding every candidate’s life situation, their deeper needs and motivations – isn’t that a huge waste of time and energy?
Fair question. This kind of hiring is more time-consuming upfront.
We all know that placement fees for technical specialists in IT can easily run between €10k and €80k. Building an efficient in-house recruiting process helps reduce those costs, but not dramatically. The real expense isn’t the fees – it’s the attention and energy of your recruiting team.
The more candidates you process, the more time you need. But at the end, you only need one or two people for any given role. Most of that effort gets wasted by the general rule of any funnel: conversion rate is never 100%.
So the optimization question becomes: how do we spend less time on candidates who won’t fit, and more time on the ones who will?
Communicate the ugly stuff early#
Here’s the counterintuitive move that actually worked: we started communicating problems upfront.
The inconsistencies within the company. The potential stress factors. The unpleasant things that candidates usually don’t discover until they’re already employed.
I know this goes against typical HR and brand policy, especially for well-established enterprises. Companies don’t like revealing ugly truths – that’s exactly what brand policy exists to cover up, right?
But the reality is: the candidate you hire will face those problems anyway. If your goal is to hire the best talent rather than save face, it’s better to communicate potential struggles as early as possible.
Not every candidate is willing to deal with a slightly toxic system architect they’ll have to collaborate with. Not everyone wants to spend their days refactoring a massive legacy data ingestion pipeline that hasn’t been documented in a decade. Nobody loves that stuff – and being upfront about it automatically filters out people who aren’t a good fit.
Yes, candidate pool will shrink. But we’re not chasing numbers. We need the best-fitting candidate, not the most candidates.
The metric of “how many people we interviewed” is irrelevant here. The MLOps engineer isn’t a high-turnover contact center role. Since less-suitable candidates filter themselves out early, your recruiting team can devote more time and attention to the ones who might actually work out. That’s where you can go deep enough to understand whether there’s a real fit – not just professionally, but in terms of values and tech team culture.
Three principles for hiring data engineers with high retention#
After years of iterating on this, here’s what I’ve landed on for hiring and retaining top data and AI talent:
First, sell your company, strategy, and vision to candidates like you’d sell your product. This isn’t manipulation – it’s communication. Great candidates deserve to understand the opportunity fully.
Second, only do this if you understand that what you’re selling actually solves their problem. Take the time to discover what they’re really looking for. If there’s no genuine match, be honest about it early. This methodology takes effort, but it pays off in retention.
Third, communicate difficulties upfront. Make sure the challenges are taken into account before candidates invest more time, and before your recruiting team devotes attention to the wrong people.
This approach takes more intentionality than the standard “sell and hope” recruitment playbook. But the result is hires who actually know what they signed up for – and who stay engaged because the reality matches their expectations.
That’s how you build data team retention strategies that actually work. Not by selling harder, but by matching better.