Hello Reader,

Six months ago, an interviewer asking about Gen AI was a bonus round. Today it is standard. At AWS, Microsoft, Meta, JP Morgan, Verizon, and most large enterprise technology teams, Gen AI questions are showing up in every SA, FDE, and AI engineer interview regardless of the role's primary focus.

The candidates who answer these well are not the ones who have read the most blog posts. They are the ones who can connect the concepts to real architecture decisions and explain the trade-offs without getting lost in jargon.

Here are the questions coming up most often right now and the answers that actually land.

What is the difference between CPU and GPU?

The average answer

CPU is good for general purpose applications, and GPU is good for running AI ML workloads

Why average?

This answer doesn't highlight the superpowers considering this is a Gen AI interview question. Think of it like this - if interviewer asks you "What is the difference between EC2 and Lambda?", and you answer EC2 is servers/Virtual Machines on the cloud, and Lambda is Serverless, that won't be a good answer.

Good answer

A CPU or Central Processing Unit has a small number of powerful cores optimized for sequential, general-purpose tasks such as application logic, branching, database operations. A GPU or Graphical Processing Unit has thousands of simpler cores optimized for parallel workloads where you're doing the same operation on massive amounts of data simultaneously.

​
Gen AI inference is basically matrix multiplication at scale, which maps perfectly to GPU parallelism. A CPU would work through that sequentially; a GPU processes it in parallel and is orders of magnitude faster. That's why LLM training, and inference is best in GPU because that's essentially a lot of parallel processing.

Then add an AWS example to delight the interviewer - some examples of CPU EC2 instance families are M, C series, and sample GPU instance families are P4, P5.

How are you using GenAI in your day-to-day work?

The average answer

This question shows up in almost every interview now, and almost everyone gives the same answer.

• "I use GenAI to write code."
• "I use it to generate documents."
• "I have a bot that automates part of my job."

Why average?

These are bad answers. Not because they are false, but because they tell the interviewer nothing about you specifically.

The prompt is not the differentiator. Anyone can type a prompt into Claude or ChatGPT and get code back. What the interviewer is actually trying to find out is whether you know how to use your own judgment on top of the output, because at the end of the day, every one of these models is nondeterministic.

Good answer

This is the framework - name a specific real-world scenario, explain what GenAI got wrong or generated inefficiently, and explain how your own judgment corrected it.

Let's understand with examples:

Unnecessary validation logic

GenAI tends to over-engineer. If you modify an existing code, the model will often add validation checks that duplicate what already exists on another para or a program that's calling this one. Left unchecked, this bloats your codebase and makes it harder to maintain.

Catching and removing that redundant logic, and being able to explain why you removed it, shows you are thinking about long-term maintainability, not just shipping something that runs.

Infrastructure as code that is not secure by default

If you are a Solutions Architect, DevOps engineer, or cloud engineer, this example lands well. GenAI-generated CloudFormation or Terraform is inconsistent. One run might generate a security group open to the entire internet. One time it might generate in YAML, next run in JSON.

The fix is not to catch this manually every time. It is to create a skill, a reusable instruction set, that tells the model: always generate CloudFormation in YAML, never JSON, never leave a security group open to the internet, and require additional approval for any internet-facing load balancer.

Mentioning "skills" and "nondeterministic" by name in an interview signals that you are current with the latest patterns, not just the basics.

DynamoDB and the missing global secondary index

Going little deeper on this one. If you have built serverless applications, this is a strong one. "I was recently coding a serverless microservice using API Gateway, Lambda, and DynamoDB. My tables had both a primary index and a global secondary index. When I asked GenAI to add a new feature, it generated a DynamoDB scan instead of a query. A scan reads the entire table with no index, which is inefficient and expensive at scale. I reviewed the generated code, caught that it ignored the GSI, and told it explicitly which fields to query on instead."

That answer shows you know how to review AI-generated code for performance, not just correctness.

Slide generation and system diagrams that are subtly wrong

GenAI can produce a full slide deck or architecture diagram almost instantly, but it makes small factual errors that are easy to miss if you are not paying attention.

A common one: claiming spot instances save up to 75 percent, when that number actually applies to reserved instances. Spot instance savings are typically over 90 percent.

Another common one on system diagrams: showing SQS invoking SNS, when the correct direction is SNS publishing to SQS, since SNS is a one-to-many pub/sub service and SQS is a queue that receives from it, not the other way around.

Catching these small inversions and misattributions is a strong signal that you understand the actual services, not just what the tool generated about them.

This is using the same framework I mentioned. And when a candidate mentions how she used her own judgement to drive GenAI and generated a better, she delights the interviewer.

How will you cost optimize Gen AI workflow and application?

This question is getting more popular fast, driven largely by companies worried about sending proprietary data to a model provider.

The average answer

Some of the average answers I hear are:

• I will optimize prompts
• I will use cheaper models
• I will use cache
• I will reduce usage

Why average?

• Vague answers without showing architect level thinking. For example - you simply can't use cheaper models sacrificing quality
• There are different caches, need to specify to show SA depth
• These are just basic techniques, there are intermediate and advanced techniques you must know

Good answer

Understand and memorize 3-5 from the below to delight the interviewer, as well as optimize cost in real-world projects.

• Use right model for the right task. For example, for summarization you don't need to use the most powerful model, Haiku is good enough. However, for each task, use Eval to find out if model is underperforming. We can also use intelligent prompt routing which can automatically route requests to different models based on prompts
• The biggest cost factor is LLM processing the tokens. Caching helps reduce that. There are two main types of caching in Gen AI
  • Prompt Caching - this is offered by the model providers itself. If the first part of the prompt ( includes system prompt, tools, or documents) matches completely with a previously processed prompt, then LLM provider sends the cached answer
  • Semantic Caching - Whereas prompt caching needs exact match of the prompt, semantic caching can understand the underlying intent of the prompts, and if the intent of current prompt is same as a previous one, it sends the cached answer. Example: A user asking "How do I reset my password?" and a second user asking "I forgot my login credentials, what do I do?" receive the exact same cached reply without triggering a new AI generation. Semantic caching is implemented by the application. Example - Redis Caching
• Remove unnecessary MCP servers. Every connected MCP server loads all its tool definitions into context on every message. If you type a two-word prompt and wonder why the token count is already high, MCP overhead is the reason.
• This one is growing (study this on it's own for separate questions) - Utilize memory. Without memory, the model rediscovers the same context every session. With memory, the model retains summaries, preferences, and prior decisions so you pick up where you left off.
• Vector database cost sneaks up on you. OpenSearch Serverless has low latency but charges for reserved capacity even when idle. Aurora PostgreSQL with pgvector is a strong middle ground if you are already running a SQL database. The S3 vector database option is the most cost-effective for batch processing and cost-sensitive production RAG workloads, with slightly higher latency that is often within acceptable SLA ranges. Test your specific workload before committing.
• Remove unnecessary and stale data from vector databases
• Standard cloud cost practices apply to GenAI workloads the same way they apply to everything else. Enterprise discounts, reserved capacity on Bedrock, spot instances for EC2 inference, right-sizing, scaling inference endpoints to zero when idle, batch inference, and cost allocation tags by team and project.

Go crush your next interview 🙌

Keep learning and keep rocking 🚀,

Raj

P.S - If you want to get an AWS Solutions Architect job without coding or learning every AWS service, the 10th cohort for AWS SA Bootcamp is launching on Oct 17th, 12 PM ET (Eastern Time) via live workshop. This program now includes our updated Gen AI - including FDE roles! Please register below:

Here’s what you get when you show up LIVE:

1. The myths keeping most people stuck - and what actually gets you hired as an SA - I've conducted over 300 SA interviews, so I know what I'm talking about!
2. How GenAI is reshaping the SA and Gen AI roles including FDE, and the exact AI concepts (RAG, agents, MCP, eval etc.) you need to speak fluently in interviews.
3. A first look at my new product feature, built to help you practice real-world, interview-relevant hands-on work instead of copy-paste tutorials.
4. Full bootcamp breakdown for Cohort 10, plus a special offer only for live attendees.
5. My exclusive Solutions Architect framework to prep you for today's job market! But if you’re not live, you won’t get it. No second chances.

And good news - it already worked for last cohort's students who secured cloud jobs in top companies, including at AWS, Microsoft, Google, JPMorgan, Reddit, and some of them didn't even have cloud experience 💰.

Spots are limited, so don't miss it!