People Without Dirty Hands are Wrong
And I'm tired of being wrong
I was recently charmed by this video about the Cult of Done and it crystallized a deep, deep vitality laying dormant within me.
Specifically this part, and watching this sets the tone for the rest of this post.
“People without dirty hands are wrong. Doing something makes you right. Life is filled with small-minded people with narrow horizons. And they’re all trying to kill you. They’ll kill you with words like be reasonable, play it safe. And the worst, stay in your lane. You don’t have to listen to these people. You don’t have to listen to anyone. I recommend listening to the people who are building the things you love, painting the paintings you love, and writing the musical stories you love.”
-No Boilerplate
This hit me like a ton of bricks. Maybe it’s because I was in a particular mood one random Tuesday night when I clicked on this video and I was vulnerable and bored enough to let it pierce my heart. Regardless of the reasoning, the outcome remains.
I want my hands caked up in mud.
This is another one of those “shower thought moments” where all the scattered memories collide into a singularity moment. Let’s start with the past version of Sung: I used to think learning only looked like certifications.
Take a look at these badges I used to share proudly like I was about to trounce the Elite Four on my LinkedIn profile.
Let’s be clear. Each one was filled with the brim with eager and good intentions. But the dopamine hit of adding a badge to my profile eclipsed the motivation to evolve past that. I crossed a threshold where the idea of becoming a better data engineer was more important than actually being a better data engineer. It’s not because certifications are bad, it’s because I confused academic theory to be synonymous for knowing how to solve real problems (a.k.a dirty hands). Hard lesson learned: they are not the same thing because some of what I learned on these exams is NOT what people actually use in real life, even by the creators of these exams! This example hits particularly close for me as I’ve gotten lost in the certification sauce and the people I actively mentor keep thinking this is the only path to evolve as a data engineer. I don’t want them to learn what took me 5 years to realize is not a path worth taking (think: do you see software engineers brag about certifications? hint: no).
Where My Hands Meet Dirt
For 7+ years I did data engineering consulting. It wasn’t until year 4-5 where I started to actually get my hands dirty past the certification layer above. Think things like bootstrapping my way through the tooling jungle to learn: python, dbt, airflow, CI/CD, terraform, architecture diagrams, on living and breathing projects. The whole 9 yards baby. And through this, I discovered I had been suppressing a richer inner dialogue.
Before dirty hands: theater > reality
Will this certification look good on my resume?
Will this increase my social credit score on the promotion ladder?
I don’t know how to build the thing, so a tutorial should be enough, right?
I’m sure I’ll be an expert at XYZ tech once I pass the certification, right? That’s how I kind of felt after my CPA exam (false, this was me hedging)
I’m sure I’ll love using this tool after I pass the certification, right?
I’m pretty sure I can move on to learning the next thing after this certification.
After dirty hands: reality > theater
Am I solving a real problem?
Is the problem worth solving? (think: spending 10 hours to automate away a problem that manually takes 1 hour once a quarter to solve)
Is it worth the learning curve to solve this problem a custom way vs. out of the box way?
Is there enough proof that real people have solved problems with either path?
What other problems will this cause for me? (think: prescription drug commercials where it relieves eczema but makes your kidneys melt)
What makes me want to trust something that’ll go the way of open source → source available (most people don’t like this)?
Does it have strong maintainer leadership?
Does the tool I’m using require a vendor pro-serve engagement only to find out they know less than me about the tool I just bought from them? IYKYK
How do I prove people will actually use the solution I made?
Is this something I’ll be proud to share with others?
What’s beautiful about this is not that I have all the silver bullet answers under the sun. It’s that my head is now humming an inner dialogue that’s congruent with how I want to operate in the world vs. buzzing in the hopes that someone will recognize and value me (including myself). I get the joy of biasing away from self-delusion and hands holding real gems of accomplishment vs. hands waving through vapors.
Understanding Other People’s Hands
I’ve gone through this “after dirty hands” dialogue hundreds and thousands of times. These questions are primal muscle memory for me at this point. And it’s one of those things where you think, “Oh I bet everyone thinks through these questions. They must live through them too!” But I’ve learned the hard way in both data engineering consulting AND selling software AND mentoring 10+ people, that people sometimes ask these questions. Very few people answer them with their own hands. And I’m not here to moralize this observation: incentives, circumstances, and power structures are at play here.
But I’ve seen and heard more interesting things as of 2023 (paraphrased below).
“We bought too much software we don’t use.”
“We thought this other tool would be useful after a demo, but after a year of using it the idea of it was better than the reality.”
“Software purchase approval goes up to the CFO now. I better have a compelling argument to put something in front of them.”
“We hired someone who said they were experienced with XYZ tool, but it’s clear they’ve been making pull requests via ChatGPT and never double check their work”
I think other people are tired of waving their hands through vapors too.
And whether people like it or not, people have to get their hands dirty if they want to evolve. I thought this was annoying when selling software full time or mentoring people to build portfolios (even if no one else cares), but I get it. It’s necessary. And pleasantly surprisingly, I’ve never been happier to spend that time with people. It’s hard to feel regret after you solve a problem together, you look each other in the eyes, look at each others’ dirty hands, have a quiet moment of solidarity, and smirk in victory. The formalities melt away and you get something precious.
I think we can all agree it’s addicting: the charm of solving a real problem with people you enjoy.
Where Our Hands Meet Dirt, in Public
What I’m sharing thankfully isn’t too groundbreaking. Just take a look at tech influencers! Most of the people I (and probably you) enjoy and respect the most are on the front lines getting dirty hands.
What’s really nice about getting hands dirty in public is you don’t hesitate to stake your personal reputation on it, because you’ve lived through problem → solved. No one can take that away from you. The kind of thing where 3 follow-up rebuttals don’t feel intimidating because your knowledge doesn’t stop at a generic tutorial you read on Medium. People feel the empathy oozing out of your body language so much that most people don’t throw doubt at you, you usually get constructive follow up questions.
How do I get your thing to work with this integration?
Does it handle these scenarios?
How do I get this working on my machine?
How can I help? (the best question)
I’m saying this because it’s tempting for a lot of us in data engineering to hyper-optimize perception games via portfolios with the “right tech stack” to leet-coding (SQL questions) until our fingertips are numb. And don’t forget writing tutorials that stop just a half-notch above “Hello World!”
What Now?
Honestly, I think I wrote this as a pep talk for myself and wondered if I’m wasting my time building and learning in public. Because I look at my peers and the corporate culture at large and wonder if I’m a fool for tunnel visioning into actually solving problems vs. playing theater that I am. I’m glad my doubts are wrong. And for those wrestling with a similar inner dialogue, know that I see you. I want you to have the guts to be yourself and move forward, mess and all.
I want to continue getting my hands dirty in public. And it starts with the work I’ve done to evolve data-diff (compare data fast within or across databases, maintained by Datafold - a place I work at with pretty neat people) to solve problems that matter to me, and I think matter to you:
Problem: diffing data in DuckDB was slower than expected and didn’t securely work with MotherDuck (serverless DuckDB)
Problem: I was working on building time counters for concurrent data diffs during dbt development when I realized, these diffs do NOT run concurrently( a.k.a things run way slower than they need to - a more important problem to solve for than telling you how slow things are running)
Solution: Make data diffing concurrent through python’s built-in ThreadPoolExecutor
data-diff Pre Release Demo: Let's go fast y'all! - Watch Video
I promise this is only 10% humble brag. The other 90% is the lesson learned here contributing to open source. I didn’t know what problems were worth solving until I actually started living and breathing the problems that matter with my own hands. And, let me tell you something. My past 3 years of solving real problems in public has done so much for my career, people don’t even ask about my certifications anymore.
This is where you come in. If you’re itching to get your hands dirty, let’s get our hands dirty together in open source. It’s so gosh darn 🧃 juicy and worth it. Heck! I can do some of the project management like I’ve done plenty of times if the overhead labor is a friction point for you.
Just message me on LinkedIn, and we’ll evolve the conversation there.
I’ll catch up with you when my hands are dirtier. Cya!