Hello Reader,

Two weeks ago I was at the AI Engineer Conference in New York. I gave a talk on agent memory and building a second brain for AI agents, with a live demo.

The talk was fun. The hallway conversations were even better. I spent most of my time talking with Forward Deployed Engineers and engineering leaders from companies that are running AI in production today, and a clear picture came out of those conversations.

In today’s newsletter, I will share my biggest takeaways and what they mean for your career. Let’s get started with the one I hear about the most.

Takeaway 1: You Are Not Late

Every week someone tells me in my DMs that they missed the AI wave. After this conference, I can say this with confidence: the serious adoption is just getting started.

The companies doing real work are picking the stable features of AI that have been tested in production. Nobody I talked to is putting a shiny new tool into production because of a “trust me bro” benchmark on social media. Banks and insurance companies move on proof, and proof takes time.

I heard the exact same panic about cloud back in 2014 when I was at Verizon. People said the train had left. In reality, large enterprises were still moving their first few workloads, and the folks who learned cloud during those years ended up leading the big migrations later. Heck, cloud migration and modernization is still not halfway done! AI is in that same phase right now.

Takeaway 2: The Harness Is Where Quality Comes From

If there was one word I heard over and over, it was harness.

The harness is all the scaffolding you build around the model. It includes the system prompt, the tool descriptions, how much context you pass in, the agent loop, and when you use a hook instead of letting the LLM decide on a tool call. We covered hooks versus tool calls in a previous edition, and this is exactly where that knowledge pays off.

Financial companies are spending serious effort tweaking the harness to get better output from the same model. Swapping to a bigger model is expensive and slow to approve in a regulated company. Changing a tool description or tightening the context window is cheap and fast. One engineer told me their biggest accuracy jump came from rewriting tool descriptions, with zero change to the model.

Takeaway 3: Custom Evals Decide What Ships

The partner topic to harness is evals. A public benchmark tells you how a model performs on someone else’s test. A bank needs to know whether the agent gets its own dispute policy right.

So these teams build custom evals. They create a golden dataset of real questions from their domain with expected answers, then score every new version of the agent against it. Many use LLM-as-a-Judge to score the output at scale, and on AWS you can run this with Amazon Bedrock Evaluations.

Harness and evals work as a loop. You tweak the harness, you run the eval, and you compare the score against the last version. Without the eval, you are guessing whether your change made things better or worse.

Takeaway 4: Who Is Actually Thriving

This one surprised a lot of people I talked to. Let’s break it down with our favorite framework.

Bad: You assume you need a machine learning background, so you go back to studying linear algebra and training models from scratch. Or you chase every new framework that trends on social media and never finish building anything.

Good: You learn the GenAI fundamentals like RAG, agents, MCP, and agent memory. A lot of candidates do this. This is good, but it will not make you stand out.

Delight: You understand customer problems, you know system design, and you added GenAI on top of the skills you already have. This is the profile of the people who are thriving right now. Among the FDEs and architects I spoke with, roughly 99% did NOT come from an ML background. They came from backend engineering, infrastructure, networking, and solutions architecture.

What This Means for Solutions Architects

Look closely at that delightful profile. Understanding the customer problem and designing the system is the job description of a Solutions Architect. The Forward Deployed Engineer role is essentially an SA who also builds and ships.

Your years in networking, infrastructure, SAP, or QA are an advantage. You already know how enterprise systems break, how approvals work, and why reliability matters more than novelty. GenAI is a new layer on top of that foundation.

Action Steps for This Week

1. Build one thing with stable tools. Pick a single agent with Strands or a RAG app on Bedrock Knowledge Bases, and take it all the way to running on the cloud.
2. Treat your harness like code. Put your system prompt and tool descriptions in Git so you can track every change.
3. Create a small eval set. Write 20 questions from a domain you know well, with expected answers, and run them after every harness change.
4. Upgrade one interview story. Take a past project and describe how you would add a GenAI layer to it today, with the business impact.

Keep learning and keep rocking 🚀,

Raj

P.S. If you have found this newsletter helpful, and want to dive deeper...

Everything I teach on AWS, System Design, Kubernetes, and DevOps - all in one Skool classroom. Build the technical depth at your own pace, and get every new course I release: Join Here​

🎥 Free tutorials on Cloud Gen AI + interview prep on my YouTube channel: Here​

🚀 The SA Bootcamp is the full transformation - hands-on practice through Cloud Colosseum, interview prep, mock interviews, and resume reviews, and so much more...
Everything you need to land the $200k+ Solutions Architect role.

​Join the waitlist to be first in when the next cohort opens.