Hello Reader,

Six months ago, an interviewer asking about Gen AI was a bonus round. Today it is standard. At AWS, Microsoft, Meta, JP Morgan, Verizon, and most large enterprise technology teams, Gen AI questions are showing up in every SA, FDE, and AI engineer interview regardless of the role's primary focus.

The candidates who answer these well are not the ones who have read the most blog posts. They are the ones who can connect the concepts to real architecture decisions and explain the trade-offs without getting lost in jargon.

Here are the questions coming up most often right now and the answers that actually land.

How would you secure an AI Agent?

The average answer

I will use guardrails for the LLM, and policies for the tools

Why average?

This answer doesn't have the depth. You need to showcase defense in depth, and SA/FDE level thinking of end to end architecture security of the agent.

Delightful answer

Most people forget two things:

• AI Agents are generally called from applications
• AI Agents, at the end of the day, are just another cloud applications. Hence, all the cloud security best practices apply

With that in mind, let's go over the answer:

• AI Agents are generally called from an application. Assuming the application running on the cloud I'd ensure the traffic between the app and the agent is encrypted. If I am using Agentcore Runtime, it's HTTPS by default
• Next, both the app and the Agent Runtime uses IAM. We must use least privilege IAM roles, and resource policies so that appropriate apps can call appropriate agents, and appropriate agent can only call necessary tools, and memories
• Use Bedrock Guardrails to prevent prompt injection to the LLM, as well as stop LLM from putting sensitive info in the response
• Use Cedar gateway Policies in Agentcore Gateway between the agentic code and the tools to ensure no tools are compromised
• All the data used by the agent such as memory, or RAG vector store, must be encrypted at rest using Amazon KMS
• Proper authentication and authorization should be implemented between the agentic code and the tools so that only authenticated agents can invoke the tools. AgentCore gateway supports Cognito for AuthN/Z.

For each point, we have cloud agnostic part, and then I mentioned AWS specific part. You can replace this with whichever cloud you are learning.

How would you self-host an AI agent including the LLM?

This question is getting more popular fast, driven largely by companies worried about sending proprietary data to a model provider.

Good answer:

The agent code itself is straightforward. Containerize it, push it to Amazon ECR, and run it as a pod.

The model is the hard part. A model has two pieces: the model image, containing the tokenizer and configuration, and the model weights, which for an 80 billion parameter model means 80 billion floating point numbers the prompt runs through.

Delightful answer:

Say the above part and then, walk through the full stack.

• Model weights live in S3 and run on EC2 with Nvidia GPUs, or on Inferentia using compiled models for Neuron cores.
• vLLM virtualizes access to the model so it can scale under load, the same way a hypervisor virtualizes a bare metal instance into multiple EC2 instances.
• MCP servers run via FastMCP for tools.
• Memory runs on an open-source vector database like Milvus, backed by object storage on a persistent volume.
• Karpenter and horizontal pod autoscaler handle scaling the whole thing.

The balance:

Full self-hosting buys you security and control over your data. It costs you real operational complexity across every layer, from GPU provisioning to vector database management.

You do not have to go all in either way. Plenty of teams mix and match, self-hosting models while hosting memory through AgentCore, depending on what they actually need to have more control over.

If you get these questions, make sure to crush it! I will cover more Gen AI questions in future editions.

Keep learning and keep rocking 🚀,

Raj

P.S - If you want to get an AWS Solutions Architect job without coding or learning every AWS service, the 10th cohort for AWS SA Bootcamp is launching on Oct 17th, 12 PM ET (Eastern Time) via live workshop. This program now includes updated Gen AI topics - including FDE roles! Please register below:

Here’s what you get when you show up LIVE:

1. The myths keeping most people stuck - and what actually gets you hired as an SA - I've conducted over 300 SA interviews, so I know what I'm talking about!
2. How GenAI is reshaping the SA and Gen AI roles including FDE, and the exact AI concepts (RAG, agents, MCP, eval etc.) you need to speak fluently in interviews.
3. A first look at my new product feature, built to help you practice real-world, interview-relevant hands-on work instead of copy-paste tutorials.
4. Full bootcamp breakdown for Cohort 10, plus a special offer only for live attendees.
5. My exclusive Solutions Architect framework to prep you for today's job market! But if you’re not live, you won’t get it. No second chances.

And good news - it already worked for last cohort's students who secured cloud jobs in top companies, including at AWS, Microsoft, Google, JPMorgan, Reddit, and some of them didn't even have cloud experience 💰.

Spots are limited, so don't miss it!