We have all been there, something is not working.

You are not happy.

You want to put it out in the world and your manager/skip seems to be an empathetic ear around.

You decide to vent and rant. It feels great at the moment, but was it effective in actually solving the problem and what could be unintentional side effects? Let’s break it down a bit.

Complaints, galore!

It could be any of the below situations

• A project timeline is too aggressive and impractical to deliver

• A cross functional partner is too demanding.

• Another team is not prioritising your dependency and giving an impractical timeline to deliver a work that you need to move forward

• A difficult co-worker is not working well or withholding information from you

• A piece of software or infra is badly designed and the accrued tech debt is slowing you down

• A system you depend on is heavily undocumented with no usage/API information

• A team member is not pulling their weight

So on and so forth.

Ouch! These all seem like pretty gnarly problems.

Some of them may look like problems that your manager should pitch in and solve for you.

They also may look like great topics to discuss in your 1:1’s with your manager or skip level.

You may decide to vent and rant in front of your manager for 40-45 mins, thinking now you and your leaders are on the same page.

You may also feel more relaxed after venting on a topic.

But, then what?

What your leader may hear?

Did you pay any attention to how the person on the other side of the table may perceive these?

Disclaimer: This is just one of the many possibilities of how your leaders may read this situation. These intentionally adopt a devil’s advocate mindset.

Remember your leader is also human and each one of us have their own set of beliefs and preconceived biases.

It may also go in one of the below directions as well.

A cross functional partner is too demanding

[Leader] maybe, this person is not able to work with challenging situations and their negotiation and conflict resolution skills needs improvement

Another team is not prioritising your dependency and giving an impractical timeline

[Leader] Did they communicate in the right way? could their collaboration and trust building skills improve?

A difficult co-worker is not working well or withholding information from you

[Leader] Are they not able to work well with others? What else have they tried to smoothen the collaboration?

A project timeline is too aggressive and impractical to deliver

[Leader] As an experienced engineer, why are they not able to push back in the right way with data? Should I trust them with more responsibility?

A piece of software or infra is badly designed and the accrued tech debt is slowing you down

[Leader] Okay, but what can they do to solve this or improve the status quo? Did they research the problem space? Is there a design doc already?

A team member is not pulling their weight

[Leader] What have they tried already with the person? Did they dive deeper to figure out the root cause?

Get it?

After reading the possible inner monologues, you may be thinking, hold on now, the manager here is very skeptical and does not have a lot of trust in their engineer, does it?

Your leader may not necessarily think the exact same things.

They may be supportive and non judgemental and work with you to dig deeper into the problem. You’ll never know how they took it for sure but may get signals (verbal or non verbal). 🤔

Read more

Why MCP?

Before MCP (Model context protocol) and Agent skills were introduced as open standards, Developers had to write their own functions and prompts to provide LLM’s (Large language models) access into existing data sources and tools (such as API’s, databases, files). Many ask their models to generate APIs or scripts on demand to achieve some outcome and hope for the best. 🤞

You had to provide the right context and hope that the agent would discover the correct API (Application programming interfaces) function to pick. Also, from an LLM’s perspective, to identify the right resources to do a task, it had to load everything in its context window and then go from there. This was both inefficient and also wasteful in nature.

While this approach somehow worked with the occasional hallucination, in practice, it made it incrementally hard to produce systems that can be reused between developers and relied upon.

MCP makes an attempt at solving this problem.

Just like web frameworks and REST/gRPC helped standardize API development by providing a framework and structure for writing better API’s. MCP provides a similar structure on how an LLM could interact with different data sources and do tasks for you.

What does it mean in practice?

MCP allows LLMs to use capabilities from different systems such as,

1. API’s

2. Data sources and resources (files, databases),

3. Use Tools (scripts) to perform different tasks at hand.

Let’s understand this with an example.

Say,. You want your LLM to be able to provide you with helpful info about the weather before you leave your house. Assume you have an API that provides your weather information for different countries and states in the US.

How can you make it easy for your LLM to interface/use this API as you instruct it in natural english?

You can either use an MCP server exposed by this provider or create your own to get the LLM to query the weather and then return a response in natural language like English. You don’t need to know how the API is implemented under the hood, nor how the MCP server is created, but adding it to your client (IDE or a desktop/web app) provides you the capability to make use of it.

Sounds cool? Yes, it is. We’ll build this MCP server shortly for this exact use case.

Anatomy of an MCP server

What makes an MCP server?

Well, three main things really.

Tools

MCP servers expose a set of tools that your model can invoke to get a task done. What are some examples of tools? If you open tools in VS code in Github Copilot chat, you can see a few built-in tools like search files in workspace, read files in workspace, execute code and apps on your machine etc. If you can write a function to do something, it can be exposed as a tool.

More concretely, there are 3 types of tools

1. Built in – built out of the box by your IDE (Integrated Development Environment)/CLI (Command line interface) Agent. For example, see GitHub Copilot in VS Code cheat sheet for list of tools

2. MCP tools – these are third party open source tools that are built by developers, see MCP Registry for a list of the popular ones

3. Extension tools – Within VS code’s context, you get a bunch of tools when you install an extension, you can also use language model tools API to provide functionality while accessing the full range of VS code extension APIs. If curious, you can read Language Model Tool API | Visual Studio Code Extension API

Read more

AI Agents may be a bit confusing to start with because there are so many different flavors of them on different toolchains.

If you are new to the topic, you may be wondering:

What is an Agent? How are they different from chat bots like Chat GPT / Microsoft copilot / Gemini etc?

The quick dirty answer is: They can do stuff for you apart from answering questions.

Notice the emphasis on “Do”

What stuff?

Below are some common use cases in the world of Software Engineering

1. They can write code for you; it could either be source code like a UI component in a React JS web app, a backend service, an automated test using playwright, Selenium, Appium or other tools

2. They can edit existing code for you i.e. help you with refactoring code

3. They can analyse your code and explain how it works; you can ask questions and use them to explain certain snippets

4. They can even write docs for you

5. They can generate diagrams

The list is quite exhaustive and increasing and thus all the hype and excitement.

Sounds pretty magical right?

You now have the capability to insert text in English and get one of these artifacts as output: code, docs, images, videos etc. And this could be a one time ask or even a system that can generalize a bit.

Like a nice person, model and tooling ask for your permissions before doing something a bit more serious, like run a command, take over your screen and do stuff for you etc.

A word of caution:

Also, just like you won’t hand over your bank account passwords to a stranger. Please don’t trust a machine with unattended access on your personal data. Do not relinquish control. Don’t be lazy; be smart and know how to work well with technology. Anything that is sensitive in nature like your secrets (passwords, passphrases), bank details etc should be guarded. If you don’t want it in public; guard it and be careful around security and access control.

With that moral obligation off my chest, let’s dive in

For this blog, I’ll assume you have a programming background or are in technology space but even if not, the general principles are still pretty intuitive to know about.

How can Agents do so much?

The lay person answer is:

1. We provide them relevant context and knowledge in the form of docs.

2. We give them clear instructions on how to go about their business with examples of what good looks like

3. We give them access to tools and resources using Model context protocol (MCP) servers; MCP being your glorified API (Application programming interface) for Agents to interact with your favourite tools and technologies e.g. Google docs, Figma, etc

4. We now teach them fine grained skills using Agent skills such has how to execute a test, how to fetch logs and make sense of the returned data etc

5. Agents now have larger context windows and memory capabilities as well, so they can remember stuff as they execute a given request for you.

And that’s kind of it, broadly.

Types of run modes

At the moment, there are 3 main types of modes in which Agents can run:

1. Local - on your machine, in an IDE (Integrated development environment) like VSCode, PyCharm etc

2. Background - on your machine, but in a terminal via CLI (Command line interface)

3. Cloud - on a cloud VM or container, these are generally integrated with your version control (VCS) systems like Github and can work autonomously and generate a pull request (PR) for you.

Read more

Here is the upfront secret about senior engineers.

They need to show high ownership over a meaningful technical scope. The definition of scope varies based on your specialization, context or discipline, but it’s usually a product, technology or infrastructure that is deemed valuable to the business.

What do I mean by ownership?

It is the act of understanding the nuances of the problem space deeply, establishing yourself as the go-to-expert, setting the right goals and priorities, crafting a roadmap and then working backwards from customer impact.

You may push back by saying that even mid level engineers have ownership over their own frameworks, tools, infrastructure, components, app/service etc … etc …

What is so special about a Senior engineer’s ownership level here?

I think it boils down to complexity and ambiguity of the problem space. Leaders generally expect their senior engineers to be able to hold their own in their chosen domain, be able to build clarity over unknown areas and solve hard problems.

Another way of thinking about this is that you own the impact and are a stakeholder. If the project fails, you also fail with it and are accountable.

The F word here can be scary depending on how you look at it. Each failure is an opportunity to learn, reflect and get better at not making the same mistakes again. It’s anyways much better than not trying at all. Even if you fail, at least you’ve been in the race.

Let’s dive a bit deeper and understand how you can personally cultivate an ownership mindset.

Do you care in the first place?

Before reading further, ask yourself do you actually care about this?

You can be an adequate senior engineer without showing strong ownership as well; your career growth may naturally get stunted though. If you do choose that path, make sure you are really okay with it.

One natural way to have more ownership is to care; pick something that you either are intrinsically motivated and passionate about or something that solves a real need or pain point for your org and company. Such scopes are often higher risk and higher reward in nature.

Learn deeply first 🌊

You should know your space deeply first to be able to take ownership.

Learning the fine art of how to learn well, asking the right clarifying questions at the right time and not stopping until you have a pretty good mental model of the space is the right first step.

If this is not the case for you, this is the first thing to start with.

Read more

A trait that really separates great software engineers from your run of the mill ones is the clarity they have and how they can talk and convey their thoughts and ideas in very specific manner

“I didn’t have time to write a short letter, so I wrote a long one instead.” - Mark Twain

The quote captures what clarity is and why it is hard to achieve. It requires deep and clear mental models of the technology you are discussing about, and is often built on top of years of experience and practical intuition that can be developed only by being hands on.

Have you observed below?

1. Sometimes people go on a monologue during meetings and keep on speaking for 5-10 mins without saying anything of substance? It’s quite easy to ramble or even speak in uncertain terms

2. Also, often quite visible in poorly written design docs, you’ll often see docs and posts that are extremely long yet sparse in terms of high quality content. You have to really work hard to separate signal from noise in such design docs.

3. Also, long paragraphs that have tons of words, yet lack any specific new idea or original thought.

If you catch yourself doing this, stop and reflect on whether you actually know what you are talking about?

And, For my impostor syndrome afflicted friends, It’s completely okay if you don’t though, but realising this and trying to learn more so that you can compress your intuition into specific words that are information dense, void of fluff language, is a skill that will really separate you from others around you.

This is also a learned and practiced skill that gets better with time. If you are intentional about improving your verbal and written communication and are generally passionate about your area enough to go deep, chances are you’ll mostly get there.

Read more

You may have heard this before, “Your relationship with your direct manager is extremely crucial and can make or break your career”

Is this a trite statement?

Yes, let me be a bit more specific

Your manager by virtue of their title and role are in a lot of conversations and rooms which you will never be in. A core expectation for a manager is to retain and grow their people, so they are incentivised to be invested in your growth, but have you given them any reason to do so?

You may be thinking, shouldn’t they do this automatically? Yes, but they also have to manage other engineers at different levels and understand and try to fulfil their expectations as well. This is just one aspect of their job. They have to also ensure delivery happens on time to make progress towards the organization’s objectives, optics of the team, relationship with cross functional partners etc… etc …

Yes, their time is precious, and while they are managing you. You should also help them manage you, but learn how to manage up.

Should I manage my manager?

This may sound a bit counter intuitive, if you are running into this term for the first time.

Managing up is the art of how you ensure you and your manager are in sync, how you provide them all the necessary context to mentor/sponsor you effectively.

How do you do it?

Read more

Tell me if any of below scenarios seem familiar

Each annual performance year

• Do you feel massive impostor syndrome as you start hunting down slack/teams group chat/channel post messages, emails and wiki’s/docs to piece together an impact story?

• After an intense week at work, you are left wondering if you got anything actually done?

• Did your work have any impact and is it appropriate to your scope and level at work?

You seemed to be busy all the time and were multi-tasking a lot. Where did all the time go?

Well, I used to face the same problem until I started keeping a work journal to track what I’ve been working on.

I’ve found this practice recommended by many senior and principal engineers and have been following this personally for the past 7 years at 4 different companies with some really good benefits.

What are some of these magical benefits?

Well …

1. You can read your journal and reflect on how the week went and what did you build/work on?

2. Read your past work and give a better standup update to your peers and leaders

3. Create a weekly summary for your leaders

4. Roll up a weekly updates into a monthly summary for broader discussion during one on ones

5. Automatically have bunch of data points around impact for your annual performance review

6. Your major projects/impact could even be something you can update in your LinkedIn or resume when the time comes around to search for a new gig. Having crisp impact stories are really effective during behavioral interviews for senior engineer interviews

With LLM (Large language models), you can use them to analyse and get some free brainstorming/coaching from your friendly AI (Artificial intelligence) model

It only takes ~10 mins out of your workday and this simple yet effective practice gives you the peace of mind that you don’t have to remember what the heck you have been doing at work over the past year.

In this blog, Let me share my personal approach and template with you. If you like it, feel free to steal it and adapt to your own needs.

Read more

We have tons of advice on how to prepare well for interviews, how to succeed in an interview, negotiate your salary etc and get that job that you seek.

But …

What about when you finally join the company?

How do you ensure you land well in your new role and put yourself in a position to succeed?

I feel this topic has not gotten much love ❤️, and I hope to change that a little bit with this blog.

With many companies offering remote positions, I’ve found assimilating and onboarding well as a new team member is quite an art in itself. Many new employees struggle in this phase especially given that expectations from managers have increased.

I’ve worked at 7 different companies over my 14+ years of career in Software engineering and have onboarded onto multiple teams and companies of different scale. During my early career years, I mostly went with the natural flow and the vibe, but over the years I’ve refined a method out of the madness and learned a thing or two about certain practices that are quite helpful while onboarding.

Let’s get into those.

Disclaimer and caveat: I will structure this in terms of loose timeline ranges of 1st week, 2-3, 4-6 weeks etc. These are just there to give you some mental hooks around how to frame your onboarding journey in your mind, depending on your context it may take longer or shorter.

For example, If you are a senior engineer tasked with a complex cross functional problem from the get go, it may take longer. Please don’t put unnecessary pressure on yourself to go faster etc. It does not help.

Do what comes naturally to you, but acknowledge the fact that any manager generally expects a new hire to be productive and fairly independent in ~ 3 months (give or take) from the date of joining. For any exceptional cases, please align with your manager.

Your 1st week ☺️

Setup your fundamentals, meet people and leaders

There are two common scenarios when it comes to onboarding on a new team

1. You are a new employee to the company

2. You are a seasoned employee, but are onboarding on to a new team in either the same org or different org.

For this blog, I’m mostly adopting the persona of a new employee to the company as that I feel is more challenging, but the approaches and practices mentioned here remain quite similar.

Setup your basics 🐘

Let’s start with the the fundamentals, during your first week

1. Get your work equipment (laptop, phone), swag etc

2. Install tools and software required for day to day working

3. Sort out admin stuff like your documentation, setup payroll

4. Attend orientation from HR team and other peers/leaders

5. Meet and greet with people on the team and cross functional areas etc.

The first week is always exciting with tons of things going on and the best advice I can give at this stage is to focus on setting up these fundamentals well.

Don’t be in a hurry to onboard just yet. Take your time to understand the environment you are in first.

General onboarding mindset and approach 🦘

Below are a few tactical things you can do right away.

Don’t be lazy. Take notes

Please don’t rely on your memory to help you take in all sorts of random facts, procedures or context about your team/org/environment. Our brains are meant for higher order thinking and remembering all the details is not particularly your brains strong suit.

How should one take notes?

The simplest advice I can give is to go with whatever medium you are most comfortable with. Paper notes more your thing? Good, take those.

If you ask me personally, You should use a cloud drive with docs that can be easily accessed and modified or even markdown plain text notes

Ask questions and be curious

If you don’t understand something yet, don’t keep quiet. Ask clarifying questions to understand context better. Being silent when you are new on the team does not work in your favor, as it leaves everyone wondering if you actually are getting what’s being discussed.

Here is the secret, when you are so new, no one expects much from you, utilize the newbie hat and ask questions, also take the effort to note down the answers and resources shared by others and offer to update the onboarding docs FAQ sections. This is such a low hanging fruit, that I’m often surprised that more people don’t do this.

Your team will thank you since you took the time to do something no one likes to do; write good docs.

Speak to your manager and senior engineers on the team

You should have a 1:1 with your direct manager and a few senior engineers on the team who you may be working with.

Please don’t focus only on pleasantries but ask them:

1. What do they work on? Their history, journey on the team.

2. What are their biggest pain points?

3. What are the current goals the team or org is working towards?

4. What is a low hanging fruit they wish someone can help them with?

5. Who should you talk to next?

These are some example topics you can get into, you should have a sense of what people on your team work on and start peacing the parts of the puzzle.

Read more

Do you want to be a great Software Engineer? There are many things beyond knowing how to code or a technology deeply. In this blog, I break down all the habits that have served me over the years and that I’ve picked up on by observing other successful senior engineers.

I’ve been a software engineer for around 14+ years now and I feel this profession has been truly incredible in terms of learning new technologies, meeting incredibly talented people and working on a whole buffet of problems related to Testing. The feeling and rush of working together with a group of people to land a feature, capability, project with style and taste and see it being used by millions of people and create value in the world is truly unparalleled and makes it all worthwhile.

I’ve also had the joy and privilege to mentor or influence many bright upcoming engineers around me in my day jobs and on Topmate. Each one is a beautiful snowflake with their own unique habits, workflows, aspirations and world view. I still learn from them and their unique perspectives.

Over the years, I’ve made tons of personal mistakes, learned a lot of things through experience and being at the craft for this long and in this blog I wanted to share some of these habits which I wish I had adopted sooner and looking back have made all the difference in my life.

Some of these may sound obvious to you and this is not a short laundry list of checklist items to evaluate, but hang on and read the whole thing. You just might pick up something a bit more nuanced?

3 pillars for a software engineer

What are the 3 most important pillars that a software engineer should focus on?

1. ♥️Health

2. 🫂Connection with Family and friends

3. 🧑‍💻Career and growth

Wait, you may be wondering the 1st two points have nothing to do as an engineer, so why are these here?

Yes. You are right and they are at the top because as you’ll discover failing in these fails you in your career as well.

Health

Why is focussing on health so important?

Software engineers could become incredibly unhealthy 🤒 pretty fast. We sit at our desks for long hours putting an incredible amount of toll on our bodies and minds, some of us are over caffeinated on coke or coffee and binge on junk food from the vending machines a lot.

Have you observed the intense fatigue you experience as you close the end of your work day with a heavy head, tired legs and slightly burning watery eyes.

Sometimes I wonder, all I did was type away on my keyboard and talked to people, how is it that I feel so tired at the end of the day?

I’m not a lifestyle and fitness coach, but I’ve seen many software engineers neglect focussing on their health that ends up causing a lot of harm. If you also want to do this job for as long as you possibly can, you need to start taking care of the fundamentals first.

What do I mean by health?

Let’s break it down a bit, shall we?

💪 Physical health

Let’s focus on our most important asset first: our body. 💪

1. 🪑 Posture: You should learn about ergonomics and invest in getting yourself a good height adjustable table, chair, monitor, keyboard and mouse. Please don’t type away on your laptop with a hunched body. The amount of time and money you’ll spend here is negligible compared to the amount you may have to pay later in hospital bills to relieve yourself of back pain, neck pain and wrist pain etc. Here is a good downloadable PDF to know more and get started.

2. ☀ ️ Taking regular breaks: Ergonomics and having good equipment is a great first start, but your body needs movement. After every 45 mins of sitting at your desk, take a 5 min break to walk around, stretch your body. It would relieve the built up tension in your muscles and your body will really thank you.

3. 🥛 Hydration: Humans need at least 2-3 liters of water in a day. Avoid having too much coffee, tea or soft drinks. Yes, 1-2 cups of tea is perfectly okay but don’t go overboard on caffeine/coke in order to stay alert and pound away.

4. 🤸 Regular exercise: Take the time to go to the gym at least 2-3 times in a week, join a dance class or learn how to do Yoga. I personally set up calendar reminders on my personal calendar and also used Habitica to nudge me into forming these habits and do a self evaluation on if i’m on track with these or not. There is something about your digital avatar losing health because you did not follow up on your dailies.

5. 🌿 Diet and nutrition: Please learn about nutrition, chew your food and ideally prefer home cooked food over hitting the restaurants every alternate day. Learning how to cook delicious, healthy and nutritious food is literally a superpower. 🦸

🧠 Mental health

If you follow the above advice, hopefully you are investing in your body and this will also have a direct impact on your cognition, mood and energy levels.

You’ll notice the increased energy and productivity to do more, but what about your most important partner in crime: Your mind?

1. 🧘 Meditation: I started meditating using Headspace in 2018 and have had a pretty regular meditation practice followed by 10-15 mins of Yoga with surya namaskars in the morning. You can choose your own workflow/app for this but I’ve found this incredibly helpful to be more relaxed, calm and mindful. Yes, I’m still a work in progress but I really enjoy learning new things about the art and science of mindfulness. I’ve found software engineering could burn you out pretty fast and having a healthy mind is table stakes

2. 📓 Reflection: Set up some regular time blocks to reflect on your day, activities and growth. Take the time to organize your thoughts better, reframe challenges into opportunities and more and cultivate a growth mindset

3. 🙏 Gratitude: You only get 1️⃣ one life, and it’s your own. Please don’t compare yourself to others journeys and highlight reels. I occasionally take some time to remind myself of things that I’m grateful for before a meal or sleeping and the things that are going well in my life and trust me it helps. Of course, I’ve blocked all social media app notifications and I’m embarrassed by the joy of missing out (JOMO) instead of fear of missing out (FOMO).

4. ✈️ Travel: Please don’t be glued to your office and desk 365 days a year. The world is vast with interesting experiences and you should plan regular breaks and travel with your friends and family to unwind and not think about your work/office. Yes, working on your career and success is important, but it’s not the most important thing you’ll do with your life. There is a life beyond the tech-verse, explore it!

😴 Good Sleep

This is an area you should never neglect for anything.

We all have different circadian rhythms. Whether you are a morning person or night owl, getting 7-8 hours of good quality sleep is essential.

Invest in building good habits like slowing down with a calm and relaxing night routine wherein you ensure your body is primed to sleep and wake at consistent times. Your body will thank you.

Your projects and tasks will still be there when you wake up, but if you don’t sleep well, your body will not be able to repair itself and you’ll pay the toll the next day anyways so cutting on it really does not help.

💰 Financial health

I consider it to also be part of your health.

Why?

My logic is simple. If you don’t have good financial habits and money, you’ll not be able to think of anything else. Your career success can help you acquire money, but saving and growing it is a different skill.

This is also a topic they don’t teach well in school but is extremely important. Investing in living a life of simplicity and moderation where you live way below your means while you invest in your career growth and invest smartly may be the single most important thing to learn early on in your career.

How I wish I knew more about the financial traps early on in my career and not miss out on the wonders of compounding. It’s never too late to start and the best day to start your personal finance journey is today. I’ve found Personal finance modules in Zerodha Varsity and Zero1 by Zerodha - YouTube to be incredibly useful.

⌛ Working hours

This one has become unnecessarily controversial

Working 10-14 hours daily or 996 culture has been way over glorified by hustle/startup culture with promises of exponential growth and riches. Your body may be able to sustain itself for short bursts of such an environment but this is definitely not scalable over a long time period.

This is also truly a personal decision and I will not tell you how to live your life, but personally I’ve found 8-9 hours of focussed, well planned work a day to be more than enough for my brain to sustain. If I can do this consistently for 5 days with some good outcomes, impact or progress at work then I can head into the weekend feeling like I spent my time well.

Why does discipline this matter?

I don’t wish to have a 5 year career sprint, retire and sip martinis on a beach somewhere. I realize that is aspirational for my people and I have no issues if that’s what you are striving for.

I want to work for as long as I possibly can, have the desire and love for the craft, and this is directly proportional to me having a holistic life. Thus, I need time to focus on all the other stuff I’ve mentioned above. If I work 14 hours a day, when would I realistically have the time for other stuff?

So yeah, healthy boundaries are incredibly important to me.

There are some exceptions to this as well.

1. Your production service may be down

2. Your app may be crashing leading to direct revenue loss,

3. or you may be a new employee with no trust or street cred yet.

In such cases you should definitely try to help your company get over the issue in the most safe, efficient way possible or spend a bit extra time ramping up and being productive. But don’t make it a long term habit.

So let’s summarise health shall we?

Please focus on below core priorities.

1. Your body

2. Your mind

3. Your sleep

4. Your finances

and you’ll be fine … Well, mostly! 😏🤷

🤝 Connection

Why are you working as a software engineer?

It could be because of any of the below:

• Do you have a deep love for the craft?

• You are a technology nerd who has passion for learning new things,

• You love solving hard technical problems

• You enjoy developing yourself as an engineer and professional

• You aspire to climb the career ladder

• You want to genuinely help your product and company win the market

• … etc … etc

All of the above are great ideals to strive for and what makes this job fun and rewarding to spend 8-9 hours of your weekdays

A more pragmatic reason for why you may want to work for someone else is also to earn enough money to support yourself and your family. So in the end, it does boil down to social connection doesn’t it?

If you neglect your social connections in the pursuit of career success and meaning, then does the end justify the means?

🏡 Make intentional family time

Spending quality intentional time with your partner, children, parents, siblings and other relatives can be incredibly rewarding.

Your family is probably the biggest area for you to invest your time, attention and care in. Over your career, you will probably work for multiple employers and meet many interesting people in your life that you can call friends. However, at the end of it all. Family is what will hopefully stick with you in all your good and bad times.

A popular anecdote that I often copy and repeat is:

You will not remember code, powerpoint presentations, or projects on your death bed but you’ll definitely cherish the connections you made, experiences you lived and memories you created with your family and friends. These relations compound with time as well if you nurture them.

We humans are social creatures and we need these connections to not only live life well, but they also create the necessary support structure for us to thrive at work as well.

So please, give them your time and full attention. They deserve it.

🍻 Friends time

Friendships and bonds are amazing. These are relations you nurture not out of obligation but out of choice and as such they can be incredibly precious.

Friends support you, give advice, laugh with you, act like free therapists and are generally damn fun to spend time with.

So, take the time to stay in touch and connect with your childhood, college friends, meet prior colleagues, mentors that hopefully become friends and connections. Attend mixers and work events and be nice to folks around you.

Engineers can sometimes have a pretty myopic and dystopian view of other engineers in their vicinity, sometimes treating them as competition. However, for the most part, everyone is mostly trying their best to explore their own life’s stories and your workplace should not be a zero sum game.

You can choose to be additive and make some cool friends in the process. Please bear in mind that it can’t always be sunshine and rainbows, but you can develop genuine connections and friendships with people at work. If you choose to take the humane approach, you’ll often find this network that you build may be your biggest takeaway from a job or role far exceeding returns earned from your take home pay as well.

🏢 Career

Okay, 2000 words in, give or take and we haven’t spoken anything about your actual career as a software engineer.

Where the hell is this blog going? 😕

If you’ve been a patient reader so far. Thank you! Give yourself a pat in the back.

I think you must have realised by now that your health and connection with others is a super important lever 🧗 in your hand, pretty vital to your career success and overall well being.

If these areas are sorted, you may literally be able to move mountains ⛰️ at work. 😉

If not, ahh.. Life is going to be tough. Truly your choice. I mean it!

With that said, Shall we look into what good habits for a Software engineer look like?

😌 Deep Focus blocks

Hopefully you are no stranger to The Flow

The magical state of mind when ideas, code, docs all flow in a perfect stream. Where the world reduces to your monitor, keyboard and IDE/ doc/sheet editor and nothing else matters except the problem you are solving one step at a time. Losing track of time in such a state is quite common and software engineers are addicted to this state. This may be the real reason you fell in love with the craft all those years ago.

However, do you have the time at work?

What about the 4 back to back meetings with a 30 mins delay in between, 10 DM’s and constant ping and notifications you have on your Teams/slack that you absolutely must respond to right away?

Yes, this is the modern workplace and we all suffer from some degree of ADHD (Attention deficit hyperactivity disorder). Welcome to the struggle!

But, there is a better way and it’s actually quite simple.

Let me break it down for you.

1. Find a quiet place at home with a workstation, a focus room at work or in the worst case noise cancelling headphones in your open office.

2. Block 2 hours calendar blocks on your calendar. Anything less is not really useful. 30 mins to 1 hour does not count.

3. Turn off notifications on your email / instant messaging clients / mobile phone

4. Take 10 mins to get into the zone

5. And then go and work on the problem that you really care about with impact.

If you can get 2 such blocks in a day, congratulations 🎊 it’s a day well spent at work.

If not even one 😞, you need to take a hard look at what you’ve been doing at work all this time.

Stop trying to look busy, and actually be busy producing meaningful and impactful work.

📵Turn off notifications

Psst! Do you know the big secret?

You can pretty much turn off notifications from devices/apps constantly stealing your attention every moment.

The world actually can wait for a couple of hours. We often give ourselves too much self importance. In the worst case, if something is truly burning you just might get a call and then it’s fine to break your focus block and go solve that urgent and important thing.

📔 Work journal

Do you ever wonder:

1. What the hell have I been doing for the whole day and actually got done? Yet feel totally drained and exhausted?

2. Does this compound badly to you having to look at your past emails, teams/slack posts when the time comes for you to write your annual review?

3. Does your boss need to constantly pull you aside for that quick update?

Yes, all of these are real problems.

It’s also not possible for you to have a perfect recall and memory of everything you did.

Maintaining a work journal is very important and helps solve these problems.

What does it look like in practice?

Glad you asked, Say you want to start this practice from October month

1. Create a doc and put it in your cloud drive (GDrive, OneDrive, Dropbox etc)

2. Add title of October 2025

3. Add H2 of Week 1 - Mon, 6th Oct - Fri, 10 Oct

4. Add H3 with Mon, Tue, Wed, Thu, Fri

5. Add H4 with your current project title, say Project A, Project B, Meetings etc

6. Take 5 mins at the end of your day or after you’ve completed something meaningful and add a bullet point under these

7. When the week is done, summarize your impact under each project

8. When the month is done, summarize these weekly summaries into a monthly one

9. And voila, you have a pretty good, detailed reflection of how your time was spent and what the hell did you actually do

10. If you did something truly outstanding, put it in a brag doc and share it with your manager

This practice has few really good benefits

1. You can use this for self reflection, if the points are thin or the work is mostly busy work, it may be time to carve out intentional time to work on the big meaty problems

2. You can refer to this before standups and give clear, concise updates and sound cool while doing it.

3. You can share this monthly summary with your manager and ask for feedback on what else you should be doing

4. Annual reviews don’t seem that intimidating

I’ve seen this practice advocated for by many Principal engineers and have been following this for the past 5 years. It does help!

The only barrier to this is laziness and procrastination, but hey. There is nothing called a free lunch. You may also be tempted to use an LLM (Large Language model) for some of this work and sure you can use it in a way that works for you. I’ve found summarizing these points and reflecting on them does help me self calibrate.

🌱 Learn

Yes, you learn a ton at work.

The projects you work on, that legacy framework or stack, one off tidbits from a Teams/slack message from that Principal or Senior engineer you admire, hallway conversations and all of these are great ways to grow.

But …

You still need to carve intentional learning time for yourself

Why is this important? 🤔

If you take the time to learn a topic in good detail, it helps you develop solid taste in that ecosystem and often awareness and knowledge of a technology helps in you making an educated design choice.

It’s hard to create this dedicated time during working hours since you have couple of projects running in parallel

So block time to learn outside work, this can be either morning or evening or nights or weekends, it depends when you can create that time for you, but regardless of whatever time you block on the calendar, make sure you stick to this. Consistency really is the key

📓 Notetaking

I don’t care what tool you use but make sure you have structured notes on different topics somewhere.

Ideally synced and backed up in the cloud and searchable and editable on all your devices – laptop, mobile or tablet

Every time you learn something new related to a topic, come back and take 5-10 minutes to update your notes. Learned a new subcommand in git that worked for this niche usecase, come back and update. Discovered a new cool python module, you know what to do 😉

You’ll notice after some years, you will start using this as your second brain and you can very quickly recall good practices, bespoke approaches to solving certain problems and appear like a total wizard to the folks around you. Your knowledge will compound with each year and you’ll see the difference in yourself as you are not struggling with fundamentals, but building on top of these.

✍️ Writing is the new super power

Good writing is clear thinking made visible - Bill Wheeler

If you can strip away distractions and put down your thoughts in a doc in a concise and clear manner, it would really set you apart from your colleagues.

It’s no secret that a large part of software engineering is to communicate your ideas to other people, and influence without authority

Meetings are one expensive high bandwidth way of doing so, but often drains everyone.

Being able to write down your ideas on a doc, post or thread can help others participate or review and give you feedback on their own time.

Sharing my own personal journey, I started blogging in 2018 and in my opinion this was the inflection point that had led me to become a much better writer and communicator. This not only helped me in building a brand on social media but also immensely at work, as I could explain my ideas/thoughts clearly to different stakeholders in good writing and a positive comment from an interviewer where they went through my newsletter and blog to see what i’ve been up to outside work does not hurt my chances.

This is a core skill for Senior engineers and with AI models working primarily on text and instructions, being able to write down a project, specifications in clear English may just be as important as learning to code.

And that’s all folks.

I hope I left you with some ideas and approaches that help you in your software engineering career.

Did I miss any other good habits to develop? Please drop in your thoughts in the comments to share them with others and if you got something valuable from this, please share it with your friends and colleagues.

Enjoyed reading this?

Subscribe to the newsletter and YouTube channel (@automationhacks) for more insights ⚡ into software testing and automation, or connect on topmate for a personalized 1:1 session.

Disclosure: I’ll earn a small commission if you decide to purchase some of the educative.io text courses linked in the blog for your learning and growth. Their system design courses and grokking coding interview courses are arguably quite helpful for interview prep.

Firstly, some news! 🎊 I recently started my new gig at Microsoft Bangalore as a Senior Software Engineer in Teams Mobile Engineering Systems team.

This team is focused on improving developer productivity by providing the tools, frameworks and infrastructure required by Teams mobile developers to land their changes in a safe and predictable manner and optimize the build test deploy loops.

In this blog, I demystify and give you an idea on how the Interview process was like at Microsoft and share my perspective on how to prepare well by offering your some tactical tips

I previously wrote a post about how I got a job at Meta London and this blog would mostly build on top of that, it may be a good idea to read that first and come back to get additional insights on how the process is different here. Go ahead, I'll wait.

The process as you may have already imagined is pretty standard in larger tech companies but I did notice some nuances that were unique to the Microsoft interview loop.

Broadly it looks like below:

1. Getting the interview

2. Coding round

3. Domain round

4. System design round

5. Behavioral/Leadership round

Let’s break it down a bit.

Disclaimer: This blog would give you a rough outline of the process, how to prep and the mindset to have as you approach the interview loop. Some of this is already published in the public domain by Microsoft How we hire and Interview tips. I will not be discussing any questions asked or answers given to respect the company NDA.

Getting that interview

How did I discover this position?

I mostly used LinkedIn search with a custom query to pick up roles/descriptions that seemed like a fit for my interests and skill set

You can paste a query like below in the search box under Jobs and replace it with keywords relevant to your specific industry

((SDET OR "Automation" OR "Test" OR "Quality" OR "QA") AND (Senior OR Lead OR Staff OR Principal)) OR

((Test OR Quality OR "QA" OR "Quality Assurance" OR "QE" OR "Automation") AND ("Manager" OR "Head" OR "Director"))

As you can see, I cast a pretty wide net. One of the perks of being a senior engineer in the industry with decades plus of experience.

This query led me to this position on a fine day.

I could see some strong overlap with my past experience and skills and also felt there were few areas which were going to be huge learning opportunities for me so targeting this position was a no brainer.

I then used LinkedIn network to find 1st level connections in Microsoft and reached out to someone I knew and trusted for a referral.

I tailored my resume and sent it off to my contact

And then I waited …

After getting the referral, I did not hear back for a couple of weeks and reached out to my referrer. Luckily they were able to follow up internally with the hiring manager to try to get the profile prioritised.

It helped that this position was for the same team my referrer worked at so a direct line to the reporting manager did make it easier.

Your mileage really may vary here but having someone who can advocate for you internally helps a lot.

⚡Tip: The obvious tip is to invest in building your network. LinkedIn is a good place to do so and there are nuances on how to do this well. The potential gains are opportunities not just for your career growth but also learning and developing yourself as an engineer.

Read more

If you are searching for a new job, one aspect you should be prepared for is handling salary negotiations in a professional and prepared manner.

If you perform well in this phase of the interview loop, you could easily walk away with a better total compensation, which means more income for your savings, future, and family.

As engineers, quite often we feel awkward asking for more, or we don’t know the right way to handle these conversations, leading to suboptimal results. Many engineers don’t even ask, thinking they will come across as greedy and fearing that the job offer may be rescinded.

However, you are not doing any favors to yourself by thinking this way.

In this blog, I’ll help break down principles and tactics from my personal experience and provide you with actionable tips on how you could negotiate your salary the right way.

🗼 4 pillars of negotiation

There are a few key principles in doing salary negotiations the right way. Let’s understand what these are, and then I’ll help you with some helpful simulation prompts to practice handling different situations

⛰️ Get some leverage

If you are negotiating for a better salary, you should understand what leverage you have:

Understanding what leverage you have would help you know how much you can ask or how hard you can negotiate.

Self-reflection on the below would help you self-analyse if you have leverage:

• Do you have a high current compensation that could be used as an anchor point?

• Do you have a competing offer?

• Are you an immediate joiner, or can you join faster than other candidates?

• Do you bring skills and experience to the table that other candidates don’t, or are you a better fit for the role?

• Do you come with a strong referral from an existing employee?

• Did you nail the interview process and come off as a strong candidate?

📊 Always do your market research.

Many candidates appear for interviews without clarifying basic things about the role.

• How long has this role been open?

• Why is the role open?

• How many candidates are in the pipeline?

• What level does this role play in the organization's career framework?

• What is fair pay for this role in this company or roles at similar levels in other companies in the same region or economic zone?

• Who is the final decision maker for this position?

When you speak to the recruiter or hiring manager, clarifying the above questions helps you further in salary negotiations.

How?

Let’s say you are the only candidate in the pipeline and do well, the company may consider paying you a bit more to close the position.

If you know who the final decision maker is, you can leverage this information to request the recruiter to reach out to them once during final negotiations.

Understanding how much other people are making in similar roles could be used during negotiations to provide actual data points and ask for a matching offer.

😶 Don’t reveal your numbers until they make an offer

During initial conversations and also during the interview process, the recruiter, leadership, or hiring manager may ask you for below information.

1. Current or Last drawn compensation

2. Salary expectations

3. Competing offer amounts

You might be expecting this, or this may be a surprise, but revealing this information early in the process does not work in your favor. I’ll discuss some common strategies that you can use to politely deflect.

What’s the harm in being transparent with this information?

• Outside range:

  • If this is the first conversation you are having with the recruiter/hiring manager, and you are outside the range, the recruiter will likely move on to other candidates and drop your profile. You may be okay with this, but this also means you’ll have fewer opportunities to talk to engineers and leaders in other companies. Many times, once the company understands what you bring to the table, they are more flexible on compensation later on.

  • If you reveal these during the interview process and say you are either at the top of their range or outside their range, they may consider offering the job to another candidate who may be within the range and may also be a strong performer.

• Lowball offer:

  • Say your current compensation is 20 Lakhs per annum (LPA), and the company can easily pay 35 LPA for the role. The moment you reveal your current compensation early in the process, the company may give you a 30% hike and offer you 26 LPA. Very few recruiters/hiring managers would pay you the fair starting range.

  • Also, revealing the competing offer amount before the company has offered you may create a lowball scenario. Say you have a competing offer of 28 LPA while the company range starts from 35 LPA. The recruiter may say, We can match the offer and do a little better and offer you 29 LPA. You may think you are getting a better offer, but in reality, you are leaving around 6 LPA on the table.

You may think recruiters and hiring managers are evil for doing this, but this is just how this process works. It’s not personal; if they have an opportunity to save the company some money, they will most certainly take that.

You should anticipate that they will ask you for this information at different stages; it’s their job, you should let them do their job, and ensure you also handle this in a respectful, gracious manner.

You should try to ensure you reach the offer stage and let them make the offer first.

Do this, and you will increase your chances of maximizing your total compensation.

🪫 Don’t set a low anchor

Assume you’ve done some back-and-forths with the recruiter or hiring manager, and they keep pressing for you to reveal your salary information.

Some candidates don’t have a clear number in mind and ask for a lesser amount or a very small increment.

In either scenario, you, as the candidates, end up walking with a lesser amount.

You should always research what salary range the company can offer and the ideal figure you are looking for, and go at least 20-25% higher than that. This ensures you have a high anchor, and the recruiter may attempt to meet you somewhere in the middle.

This strategy does not always work, you have to be mindful of when you can do this versus not.

For example, if you know the company can offer you 25 LPA, asking for 50 LPA will not work out.

🏋️ Practice simulations

The above pillar and advice sound great; however, in reality, the conversations can be very dynamic, and it's a good idea to have some mental hooks on how to respond to different situations.

🤝 Before the offer

Let’s walk through some situations and how you can respond to these.

Remember the key goal here: don’t provide any numbers early on, and try to deflect until you reach the offer stage.

Any numbers you provide at this stage do not work in your favor.

What if they remove you from the pipeline because you did not provide salary information?

🎊 Congratulations, you’ve dodged a huge red flag and bullet. Any recruiter or company that does not have the basic flexibility and plays this hardball early on would probably not be the environment that you would want to work in anyway.

🦜 Initial recruiter conversation

* Smith – imaginary recruiter in this role play

* Neo - best damn candidate on the planet 😉

🪪 Salary expectations?

You would want to convey that you are open and negotiable at this stage and interested in evaluating the opportunity holistically. This sets the right early tone for the process, where the recruiter knows you are savvy.

Recruiter: What are your salary expectations?

Candidate:

1. Hey Smith^*, I’m open and negotiable for the right opportunity

2. (If they press you or follow up again) I’m excited to learn more about the position, meet the team, and understand the challenges and opportunities for me to make an impact, and I'm happy to discuss compensation once we’ve done those

↕️ Tell me your expected range?

Recruiter: Can you provide me with the range or number that you would be comfortable with?

Candidate: Smith, can you tell me a bit more about the salary range for this role?

If they press again.

Recruiter: We need this data to move forward

Candidate: I understand and appreciate your need to get my salary expectations. At this point, I want to learn more about the role, meet the team, and if it's a fit, then happy to discuss salary expectations.

💰 Let them make the offer.

If you’ve done well in the interview process and the recruiter reaches out and lets you know that they are willing to offer and asks about your expectations

Recruiter: Congratulations, Neo. We have decided to offer you and I need your last drawn compensation and expectations to roll out the offer.

Candidate: Smith, if your company believes I’m the right fit, I would love to see an offer in writing and discuss how it aligns with my needs

If they press more, you should push back and understand the rationale.

Recruiter: At this stage, this is a mandatory part of the process to get these numbers

Candidate: Mandatory part?

If they confirm they need the information and cannot proceed, you can push back with

Recruiter: I’ll need this information to roll out the offer; it's part of our process

Candidate: Smith, I just want to understand, has there been a situation where the candidate didn’t provide this information at this stage

Recruiter: This is a stopping point

Candidate: I would need to think about it. I prefer for companies to present the offer first. I don’t want to lose the opportunity, but I would need to think about what those numbers need to be …

And then pause and see if this forces Smith to cave in and come up with the offer.

🔱 Set a high anchor

If you have a tough recruiter at hand, and they are not willing to budge without getting your salary information

Recruiter: Unfortunately, we’ve reached a stopping point. We have other candidates in the pipeline who have provided salary, and I’ve been given a directive from leadership that we need to get this information before

If you want the job, set the anchor high. At least 20% higher than the number that you want.

If you get a surprise reaction from the recruiter

Recruiter: This number is high, this is way out of our range, and I wish you had discussed this before (some emotional reaction)

Candidate:

• I want to thank you again for all your time and effort. As I said earlier. I’m open and negotiable for the right opportunity.

• This is about the number I’m looking for, but I would like to see an offer from your company and see how it can make sense for both of us, because I’m interested, and it sounds like you guys are as well.

🏁 After the offer

If they make a verbal offer, Congratulations, you won the first battle. Get an offer in writing first.

Also, check if there is a deadline.

❌ Don’t say:

Is this negotiable

✅ Instead, say:

Thank you! I’m so excited to be at this stage. I’d like a day or two to discuss this offer that you’ve put on the table for me with my family and get back to you.

You should ideally do the negotiation over a phone call and not via email, as this allows you the opportunity to observe your recruiter's tone and speech to decide where the flexibility lies.

⁉️ Ask open-ended questions

Start by asking open-ended questions.

• Is there flexibility to move the base salary up?

• Is there flexibility to get more stock?

• Is there flexibility to get a sign-on bonus?

🫰 Present your ask

You should know your numbers at the top of your mind before you do this. You can start with

“Based on the value, I believe I’ve demonstrated, and based on my research and what you know about the organization. I’m looking for X amount as total compensation.”

Always go a little high, so that you allow them to meet you in the middle.

If they say the offer is not negotiable, and you know they are not the decision maker, you can say

“I would appreciate it if you could go back and ask on my behalf.”

📐 Know your numbers

If they say you are at the top of the base pay band, you can say,

“Is there potential to get more stock or a sign-on bonus?”

Here, your research matters. You should have the data as a link or screenshot that proves this is a number for this role, company, and level.

❎ General fallbacks

If they ask something that you don’t have an answer to, you can always say something like:

• Let me think about it.

• Can you help me understand where this question is coming from? (in a nice tone)

If they give you an ultimatum like

1. “If I meet these, would you accept the offer on the spot”?

2. “I’ll need a handshake from you now, so that you’ll agree to these no's and sign the offer.”

You should not accept ultimatums, You can say something like:

“It's something I’d strongly consider, I’m happy to be at this stage. It's something that I’d need to discuss with my family.”

Negotiate everything at the same time.

• Get a timeline from the recruiter – when can I expect an answer?

• Do a lot of research – anticipate back and forth with a recruiter.

🎨 Get creative

If they are not giving what you want, you can get creative and see what else can be negotiable. Sometimes, companies have flexibility on these aspects, which may make an impact on your quality of life at the company.

• If the bonus is going to be prorated, can I get the bonus amount for the full year?

• Can I accrue vacation days immediately?

• Can I get an extra 2 weeks of vacation?

Conclusion

I’ll leave you with this thought.

“You can be uncomfortable for 30 mins, knowing you’ll end up with a better offer, or you can be polite, knowing very well you left money on the table”

The choice is yours, my friend!

Let me know if you would like to do a mock interview around these lines.

❤️Loved reading this? Please subscribe to the newsletter and YouTube channel (@automationhacks) for more insights into software testing and automation, or connect on topmate for a personalized 1:1 session.

Disclosure: I’ll earn a small commission if you decide to purchase some of the educative.io text courses linked in the blog for your learning and growth. Their system design courses and grokking coding interview courses are quite helpful for interview prep.

I recently completed full interview loops at multiple companies, including startups, product, and Big Tech.

I was targeting 2 major role archetypes.

1. Engineering manager - QA role expected to support a team of 6 - 12 QA engineers or SDETs

2. Senior IC (Individual contributor) or Tech lead roles focused on engineering productivity.

In this blog, I’ll share my personal experience navigating these interview loops,

If you or a friend is looking for similar opportunities, this blog should provide a good mental model on what kind of interview rounds exist and how you can orient yourself to perform well in these interviews.

Let’s go 🏃

Why target both IC and EM roles? 🤾

You may wonder why I didn't focus on just one role.

The reason is quite simple for me.

I enjoy hands-on work that delves deep into technologies, and I also love working with and mentoring people, often preferring roles that push the boundaries.

I’ve played three major variations of these roles in the past five years.

1. Hands-on Engineering Manager (EM) at Gojek

2. Senior IC focused on building test infrastructure and frameworks in a complex technology landscape at Meta.

3. Principal IC supporting a group of engineers at CRED

And I enjoyed both aspects quite well.

If you personally have a strong inclination for either IC or EM roles, it's fine to only target such roles and companies. It would also make the decision process smoother for you later on, as you can avoid doing an Apple 🍎 to Oranges 🍊comparison.

What are the major interview types? 🤔

I’ve mainly seen the following types of rounds.

We’ll divide it into 3 main types of conversations – Technical, Behavioural, and Recruiter, each with its focus and expectations.

• Technical: The main questions that this loop tries to answer are: Are you a fit technically for this role? Can you raise the engineering bar from your skills, past experiences, and attitude

• Behavioral: Do you have the right attitude, and how do you work on all the nontechnical stuff – communication, collaboration, conflict resolution

• Recruiter: Initially, to identify if you are in the ballpark vicinity of someone who can do this job, and how much it would cost the company.

Let’s form a high-level intuition first before going deeper.

Introductory call

A short 30m - 45m call with the hiring manager or recruiter to align on a high level if the role seems like a mutual fit

Technical rounds (45m to 1h 30m)

Here few rounds are unique to Big Tech companies vs startups/product companies.

Big Tech

• Coding rounds: focused on problem solving on an algorithmic or data structures problem, and your communication skills as you work through them.

• System design – design a large-scale distributed system or a core engineering system like log processing, build tools, etc.

Startups and product

• Take-home assignment: Build an automation framework on either API, web, or mobile, based on your preference, and using open-source tools and libraries.

• Technical discussion on framework design and tradeoffs: if company does take home assignments, you’ll have to explain your design choices and answer follow up questions or if is the first round, this could be a problem statement given on the spot targeting API, Web or Mobile use case, and you’ll have to explain the framework design by breaking down components

Common

• Testing acumen: Given a problem, application, or system, how would you test its different aspects, what cases can you think of, and what approach would you follow

• Domain knowledge round to focus on:

  1. CI/CD, Cloud, Testing tooling, or frameworks

  2. The domain the role expects knowledge over, for example: mocking, stubbing, integration testing, load testing, and their related tooling.

Behavioral (45m to 1h)

• Your experience, projects, or resume walkthrough: such interviews focus on what you’ve written in your resume and check, for instance, the challenges you've solved, how you handled conflict, did negotiation, what process improvements you created, etc

• Discussion with a cross-functional partner (product or developer) to evaluate how you collaborate and get work done across teams and functions (like dev, PM, business, sales, marketing, compliance, legal, etc, to name a few)

• Behavioral interviews focusing on team/stakeholder management, leadership, and how you handle different situations at work

• Hiring manager interview

• Recruiter discussion

Let’s break these different rounds down in more detail so that you have a better idea of each phase.

I’ll also provide some strategies on how you could prepare well and ace these.

Resume and referral

Before we get into the actual interview rounds, let’s briefly pay our respects to the most important first step …

Creating the opportunity.

Get a referral and thank me later 🤝

Any good interview loop usually starts with:

1. Searching for relevant openings you are interested in

2. Finding out if you have a mutual connection from your network

3. Preparing a resume tailored to this role

4. Getting a referral

5. Hopefully 🤞🏼a recruiter shortlists your resume if they see a strong fit.

Each of these steps requires patience, good writing, and timely, respectful follow-ups.

⚡Tip: Please leverage your network on social media to get a referral from someone who works at the company. I’ll do a deep dive on how to create opportunities for yourself in a future blog, as it has more nuance.

Why index heavily on referral?

Your chances increase manyfold if:

• An existing employee can refer and vouch for your skills.

• They can also advocate for you internally and help oil the communication rails between the hiring manager and the recruiter. I’ve personally observed how many times a friend has helped grease internal processes to help move the process forward, leading to better outcomes for me.

If you don’t have a trusted colleague or friend in the company you want to apply for, don’t sweat it; you can use the strategies below.

1. Cold Linked message or email someone you are connected with, but don’t know personally

2. If you find a mutual connection, then ask your connection to introduce you to them and help connect.

It’s fair that not everyone is open to referring a stranger on the internet, when someone declines or does not respond; don’t take it to heart, move on to other options.

Remember to follow up, your referrer has their workload and life to manage, and referring to you may not be on the top of their priority list; reminding them periodically could go a long way.

Again, I really cannot stress enough how building a strong network from your past companies, communities, and being active on platforms like LinkedIn, X, or teaching others could help you build an organic network in the long run.

More on this later in a future blog. I promise. 🤞🏼

Write a sharp resume

Having a neat, sharply written resume that puts your past contributions, impact, and skills in the best light is critical.

Luckily, you can leverage your friendly LLM to help with improving the structure and presentation.

🚨Please be careful not to add made-up numbers that are not grounded in reality, as it can be a real eyesore for an experienced engineer/manager and they can easily smoke this out on their bullshit meter. Be honest and transparent. Do not lie to get the job. You are not doing anybody any favors with this approach.

I can recommend The Google Resume by Gayle Laakmann McDowell for tips on a well-written resume.

Once you have your first draft, make sure to get it reviewed by a trusted peer or friend to give you feedback.

You’ll likely have blind spots or biases, and a peer review is usually very helpful to catch those ahead of time.

Introductory call 📱

Most companies start their loop with an introductory call with a hiring manager or recruiter.

They have taken a look at your resume already and want to evaluate a few things.

1. Are you genuinely interested in the role?

2. How strong of a fit are you for the role in terms of skills?

3. Are you within their budget?

They will usually give you an overview of the role, the team you’ll be working with, the company setup, and the expectations.

Generally, they will ask you to “Tell me about yourself … “ and take it from there.

As a candidate, it's a good practice to have a clear outline of your major contributions in the past 3 companies in a short 3 - 5 min intro. You should make it practical and mention points where there is a strong overlap with your demonstrated contribution/experience and the role you are applying for.

If this is an EM role, try to highlight contributions where you led a team, mentored a group of engineers, led a project E2E, or conducted hiring/performance reviews, etc.

If an IC role, highlight technical contributions you are most proud of, mentoring, and leadership without influence.

Asking an LLM to analyse a job description and your resume to find this out is a helpful preparation exercise.

The prompt could be something like

As an experienced recruiter, please find below the resume [file name] and job description text. Help me identify how closely my resume fits this role?

What are the 5 key areas where there is a strong fit and 5 areas with zero to no overlap?

Help me frame a short, 200-word, concise, bulleted introduction that can be used to answer the “Tell me about yourself” question in compelling language.

🛑Before asking the LLM, I’ll also encourage you to do this exercise yourself first and leverage your brain, because once the LLM gives you the data, you will already be biased.

Remember: You are going to be in the interview, not your LLM

Technical rounds 🧑‍💻

You can expect different types of technical rounds

Coding 🧑‍💻

BigTech companies and many high-growth scale-ups or product companies have 1-2 coding rounds focused on solving 1-2 Leetcode-style problems within 45 mins. I had previously written about this when I was preparing for my meta interview, and I would still recommend this.

Additionally, I found that instead of randomly solving problems. Focusing on Leetcode's top 150 gives you good, rounded practice.

Practice daily to ensure you are hands-on and can talk out loud while solving the problem.

System design 🌀

If this role is in a BigTech or high-growth product company, you could expect the interviewer may ask you to design a large-scale distributed system.

You can read this section of the blog that I wrote on this topic.

An additional good resource that I’ll recommend is the Hello Interview YouTube channel, which covers a lot of ground on topics that you should be aware of, and also some popular questions.

Distributed Systems 1.1: Introduction series by Martin Klepmann is also helpful to understand some fundamental concepts around distributed systems

Once you have the theory down, practice solving standard problems like Chat, Video store, E-commerce, and Payment systems. Another tip is to record yourself while answering or designing a system and watch it later to observe if you are clear while explaining your thinking.

Longer term, it’s quite beneficial to take an interest and understand how different engineering systems work under the hood, like your test runner, build system, log processing and storage, analytics, and time series database. You’ll have better confidence when you get an unfamiliar problem or system, to be able to reason from fundamentals.

I know and appreciate that the learning scope is huge and you probably cannot cover everything, but treat this like a longer-term endeavour to build knowledge about these disparate engineering systems, and not only for the imminent interview.

💡 The chances of running into this round in a startup are lower. When in doubt, it's better to clarify this with your recruiter.

Take-home assignment 🏡

Some products or startups have a round like this.

Essentially, they will give you a take-home assignment where the task is to design a test framework to achieve some flow on either a dummy application or even their public-facing app, website, or API.

If this is something you’ve done a few times at work, you should be able to write solid, clean code to create a framework that covers important areas like:

• Test configuration

• Test data management

• Designing Test Suites

• Test parallelisation

• Making API calls

• Performing UI actions on either web or mobile

• Assertions

• CI/CD

• Clean modular classes that can be extended, test reporting, etc.

It’s better to study frameworks that exist in your past roles and how they are organised, and create a diagram out of it, which you can use as a base to implement the framework.

For example, below is what a high-level framework may look like without any particular domain

You may have to explain these different layers in an interview, and having a good, solid intuition helps a lot.

Below is a sample framework showing a detailed architecture using the Java ecosystem and its tools for API testing

You should have a clear intuition about how to approach a web or mobile automation framework and related domain concepts.

You can follow the basic concepts in this course that I created on Test Automation University on API automation framework in Python.

In general, be curious and learn how things work under the hood.

Technical discussion on frameworks 💬

If you created a framework and shared the code with the interview team, usually the next round would focus more on you walking the interviewers through:

• How did you design the framework?

• What tradeoffs did you consider while deciding on different tools or libraries, or layers in your framework?

You should practice this beforehand using an online whiteboard so that you can clearly explain everything on the day.

Remember: No one is looking for perfect code, but it should be of good quality.

The panel may ask you to implement a follow-up or explain how you would modify it to support new features or use cases.

If there were no take-home assignment, then the interviewer may ask which area you are comfortable with: API, Web, or Mobile, then ask you to describe a framework to test a sample application covering certain requirements.

They might give you scenarios like how your framework would handle third-party APIs, rate limiting, mocking, stubbing, or load on APIs

These questions are grounded in checking the limits of your knowledge, and most of these should be things you should have an understanding of if you’ve been doing this at work or paying attention to how others around you are building these components.

Testing acumen 🧠

Many companies may have this round, which aims to check how well you can test and your critical thinking skills.

Interviewer may give you either a real-world system like a parking system, an e-commerce website like Amazon, a reward and incentive system like Google Pay rewards, ride ride-hailing system like Uber, etc.

They may describe a system and then ask you how you would design test cases, automation approach, etc.

I find having a structured approach to be quite helpful in thinking through and answering these. You get bonus points if you can write these on some whiteboard and think out loud as you explain:

You should:

1. Ask clarifying questions to understand the problem, the system, and its constraints

2. Think of functional cases - positive, negative, edge first

3. Then move to Nonfunctional cases - Load, stress, volume, accessibility, usability, security, concurrency, reliability, chaos, depending on what your interviewer is more interested in.

This round helps separate a lot of good testers from mediocre ones, as this shows how wide and deep you can think and reason about a system, which is a core skill for people in testing.

Domain knowledge

Another technical format is where the interviewer may ask you to explain the architecture of the app, backend, or infrastructure that you support in your current role.

In general, practicing explaining this with a diagram is good practice, and it's great if you can explain how you tested each layer.

Any modern distributed system would usually have an API gateway, load balancer, db, queue, cache, and being able to explain this shows that you are a technical person on the team and not looking at things just from a black box perspective.

The interviewer may also go over different things in your resume and ask you follow-up questions to check your understanding of those concepts. Anything in your resume is fair game here, so make sure you only add technologies that you feel comfortable enough to answer questions on

If the role is more frontend-heavy, you can expect questions on:

• Selenium/Appium API

• How do you manage browsers/devices

• How your infrastructure scales.

• How are the different layers designed

• What common problems occur, and how do you handle those

• How do parallel or concurrent runs work? What problems do you see, and how do you design solutions for them

If it's backend-heavy, then:

• Which HTTP/gRPC/GraphQL client or library do you use

• How is the CI/CD setup designed

• How does load testing happen

• How do you test the cache, queue, and DB layers

Behavioural 🪑

If you do well on the technical interviews, you will come across some form of evaluation to check your team management, leadership, stakeholder management, conflict resolution, negotiation, and project management skills.

These are extremely important if the role involves supporting a group of engineers, and doing well on this provides a good signal to the hiring team in regards to your seniority.

You can read the previous write-up on this.

In my experience, these tend to have few variations and focus areas

Past work or resume walkthrough

Quite often, the interviewer will go through your resume and ask you to explain your past roles and ask questions along these lines.

• What role were you hired for?

• What was your most challenging or impactful project?

• A time when you had to manage conflict either with a peer, a manager, or a junior member?

• How do you mentor a team member (junior or senior)?

• How do you lead projects?

• How do you manage deadlines?

• A time when you had to push back in case of quality being subpar?

• A time when your curiosity and contributions made an outsized impact on a project or feature delivery?

• Reason for leaving

So on and so forth..

There are tons of variations of these questions, but it's important to have a good awareness of your past 3 companies at the top of your mind. I find it helpful to document these broader stories on a yearly basis so that you can use these to tell richer stories in these rounds.

It’s important to be aware of what you have mentioned in your resume and be prepared to explain your contributions. It would be a good idea to have these written out in advance and review them before you go for any behavioural interview.

Also, remember to make your examples level appropriate. If you are interviewing for a senior or staff candidate, your examples should do justice to the scope or complexity that you can expect to tackle in such roles in this job.

Collaboration with cross-functional partners (XFN)

This round checks how you work with leaders across QA, Backend, Mobile, SRE teams and business, and other functions

I’ve seen flavours of such interviews where a leader from product, design, technical program management (TPM), or sales/marketing functions may conduct a behavioural interview focusing on how well you work with other teams and functions.

It’s usually a good idea to have some strong stories around these lines prepared ahead of time. For example:

• Tell me about a time when you had a conflict with the dev team.

• How do you work with product management and business functions?

• Tell me about a time when your feedback and contribution led to better design or implementation.

• Tell me about a time when you accomplished something difficult by collaborating across team boundaries.

Team and stakeholder management

You can expect a behavioural interview along these lines, where the interviewer wants to understand how you lead a team and manage expectations from different stakeholders.

You can expect questions like

• What metrics or KPI do you use to measure the success of a team?

• What's your leadership philosophy?

• How do you mentor early and mid-career vs senior engineers?

• A time when you had to convince the team to stretch to handle an urgent and important project?

• How do you manage deadlines and project releases?

• How do you manage non-performance?

• How do you do performance reviews and reward the right individuals?

• How do you measure success?

• What's your intuition around growing people under your care?

Hiring manager

Your hiring or reporting manager may already be part of one of the previous loops, or they may also do a focused discussion.

Whether they focus more on technical or behavioral depends on the exact role specification and your performance in previous rounds.

In general, the hiring manager may check around:

• Salary expectations

• What's the reason for the change

• What are you looking for in this role/motivations for your role?

• How do you work with other members on the team?

Recruiter discussion

If all the previous rounds went well and the hiring panel decides to extend an offer, then the recruiter will reach out to you to convey their interest and then move you to the offer stage and salary negotiations.

I’ll write about this topic around negotiation in more detail, since it's a very nuanced topic in itself and often something engineers rarely practice well.

I’ll leave you with a few general guidelines that you should consider:

• Try to have competing offers in hand to put yourself in a favourable situation.

• Don’t reveal your salary expectations or last compensation EARLY in the process. Once you reach the offer stage, try to let them make you an offer first, and then go ahead with negotiations with valid data points (competing offers, market research, etc)

• Show genuine interest and be extremely gracious and kind at this stage. No recruiter or hiring manager likes a cocky overconfident candidate and remember that handling this stage poorly may also lead to you not getting an offer or in worst cases rescinding of the offer.

Conclusion

I hope this write-up provided you with a good mental model of what the different interview flavours you can expect are as you look for your next opportunity.

A lot of it is good preparation, fundamentals, and practice, and if you’ve prepared well, represented yourself well in interviews, you could be a step closer to your next dream job or role.

All the best!

You’ve got this. 🤝

❤️ Loved reading this? Please subscribe to the newsletter and YouTube channel (@automationhacks) for more ⚡insights into software testing and automation, or connect on topmate for a personalized 1:1 session.

Disclosure: I’ll earn a small commission if you decide to purchase some of the educative.io text courses linked in the blog for your learning and growth. Their system design courses and grokking coding interview courses are quite helpful for interview prep.

Github Copilot recently announced the general availability of Agent mode

Agentic AI, Agents, and MCP (Model context protocol) have been buzzwords that have filled much of the Internet and social media chatter in the past few months. Among all the hype about its capabilities, it’s hard to separate noise from signal.

In this blog, I’ll explain these technologies and how we can use them practically in our day-to-day software engineering workflow in simple, non-marketing language for real software engineers.

Let’s dive right in. 🥽

Agent mode for beginners 🤔

The usage aspect of Agent mode is similar to Edits or chat.

You specify a prompt and let the LLM do its thing, but let’s take a deeper look at these different terms, concepts, and capabilities that this new mode brings and try to form a good mental model

What is Agent mode?

And how is this different from GitHub Copilot Edits or Copilot chat?

All perfectly valid questions.

Agent mode can be thought of as a more powerful version of Copilot Edits that:

• Can use tools to achieve tasks that require more planning

• Understand and reference multiple files in your codebase without having to specify them manually

• Create and run tests with your permission

• Running commands on the terminal

It does this while putting the developer in control to review and approve different stages, such as which tools to use, accept, or reject changes, etc.

Tools here can be thought of as a function that exposes some capability from an Application, CLI, API, or data source. They can be used to achieve a certain task by leveraging the model context protocol (MCP). A simple example could be reading a file and writing a file in your own codebase.

What is MCP?

A couple of helpful analogies that I find helpful when thinking about MCP

1. MCP is like the USB-C port for AI applications

2. It supports a more plug-and-play way of working with AI, wherein you or Agent can add a relevant MCP server to make it easier for your AI model to talk to and leverage its capabilities to complete a certain task

MCP also helps reduce hallucination problems to a certain degree for LLMs (Large language models), as they now provide a more deterministic and efficient way to work with different tools and resources

The community has already picked up this idea and started producing a bunch of MCP servers for different existing applications to support LLM-driven development, along with OpenAI

What does a MCP server look like in practice, and should you care?

For the most part, discovering relevant tools and MCP is something that Agent mode itself may take care however, if you are deeply engrossed in the domain and know that what you are trying to build would require a certain specific MCP server, you could also add those while initiating the session

You can refer to the GitHub MCP server README to understand what GitHub functionality is exposed.

In practice, it looks like a function wherein the parameters are specified with their data type, and whether a field is required/optional

• get_issue - Gets the contents of an issue within a repository

  • owner: Repository owner (string, required)

  • repo: Repository name (string, required)

  • issue_number: Issue number (number, required)

• get_issue_comments - Get comments for a GitHub issue

  • owner: Repository owner (string, required)

  • repo: Repository name (string, required)

  • issue_number: Issue number (number, required)

We can do a deeper dive in MCP later, but for now, this basic intuition is good enough.

What would a workflow look like with Agent mode as a developer?

1. Prompt: Come up with an idea and a detailed prompt for what you want to achieve. – It could be an enhancement of an existing feature to add a new capability, add test coverage for a particular area, etc

2. Server: Add relevant MCP servers that provide said capabilities, or let Agent mode autodiscover it. – For example, if you are building a CI/CD feature, and you want to leverage the underlying CI tool, you can add an MCP server for that tool, like Jenkins, GitHub Actions, etc

3. Instructions: Add relevant instructions in a custom_instructions.md file to highlight rules, for example. You can say add a changelog of every change that you made to keep track of what the last change made by Agent mode was.

4. Execution: Let Agent mode go off and execute on your idea, review the suggestions for accuracy, and only merge changes that make sense.

5. Fine-tuning: Take control back and edit or refactor the code further before shipping it.

An actual pairing session with Copilot Agent mode

I paired with Copilot Agent mode to build a feature that was in my mind for some time.

Let’s go through the steps and, in the process, understand different nuances

When you launch the Agent mode window for the first time on VS Code, you’ll see a welcome message that gives you a hint about how to leverage its capabilities

You can:

1. Attach context using hash #

2. at the rate @ to chat with extensions

3. Use a forward slash (/) to use commands

Let’s ask Copilot Agent mode to build a feature in the test-infra codebase.

You can follow different changes that were created by Agent mode and my pairing in tandem in the branch failure_auto_fixes

Initial prompt and setup ⌨️

Let’s set up a session

We’ll copy and paste the below prompt in the chat window

As a Java programmer, I want you to implement a developer productivity feature.

I want you to be able to connect to the report portal instance running on localhost:8080 on this machine and figure out the reason for a failing test from the stack trace, and then suggest a fix for this in the tests under `src/test/java/io/automationhacks/testinfra/reqres` package.

This should be built in a scalable manner such that this auto-analysis can run on every test suite run or at a scheduled time.

The report portal has an MCP server here:

https://github.com/reportportal/reportportal-mcp-server

API docs can be found here

https://developers.reportportal.io/api-docs/service-api/get-test-items

And select Agent from the drop-down to indicate you want to use Agent mode

I additionally want Copilot to have a set of rules while working on the problem.

I created a custom_instructions.md file and attached it to the context

# Instructions

## Rules

- After making a change, update the ai_changes/changelog.md file with what change was made with a timestamp to make it easy to follow

- Keep the code clean, modular, and extensible for future changes

- Prepare a plan before executing and document that in a ai_changes/plan.md file. Also, capture the prompt that was provided by the developer to trigger that change

- Don't hallucinate or make up stuff.

- Always check your work before completing the task

📃 Planning

Copilot Agent mode first breaks the problem down into:

1. Requirements

2. Research and setup

3. Created an implementation plan

4. Dove into implementing this plan.

During this process, tools were also used to check if the file exists in the codebase.

Wherever details are not provided in the prompt, it will proceed to make reasonable assumptions

You’ll also notice the Agent mode periodically prompts you as the user to decide if the direction in which it is proceeding is making sense or not.

You can decide to pause and refine your prompts or give additional instructions to nudge it in a particular direction, or let it continue.

You can review changes made at each phase by clicking on the file and seeing how many lines were added or removed. For exampl,e ReportPortalClient.java +69 -1 indicates 69 lines added and 1 removed

🛠️ Checking and fixing compilation issues

For this problem, Copilot identified the correct components, so I just let it continue onwards

It did a couple of interesting things in this phase

1. It reads existing files to determine if the required libraries are present

2. After finishing the initial implementation, it also checks the written code to verify if there were any problems. If it detects a compilation error, it makes fixes and repeats this till the code compilation or building works fine.

📈 Drives improvements

It’s already impressive that it could reach this stage without any user intervention, so I let it continue onwards.

Here, Copilot makes additional improvements by centralizing config management with a properties file and a utility class to provide getters. This is usually a good idea and something I would have done at this stage as well. It again did a round of checks and summarized all the changes it made.

🧪 Testing

Now was the moment of truth; things looked good on a high level. I asked it to run the tests and see if the solution works.

Please follow the readme instructions to run all regression tests on the terminal and validate if the above solution works, to suggest fixes when a test fails

Copilot prompted me to run a gradle command to run the tests 🤩

Sadly, most of the tests failed, and also the implemented code did not work for checking stack trace failures in ReportPortal 🤔

🤿 Debugging

Copilot, like any good junior engineer, started breaking down the failures in the logs and tried to come up with the next steps

It correctly identified that:

1. ReqRes tests failed due to 401 errors

2. ReportPortal client failed due to 400

It correctly spotted that the project name was not updated correctly in its created properties file and modified it, ran tests again

A bunch of wrong turns

At this time, Copilot made many unnecessary changes to the test files, such as:

1. Removing ReportPortal annotation code that was required.

2. Modifying the team names to something else

It also hallucinated certain values that were not present in the enum constants, leading to build failures

I discarded these changes since they did not make sense

🧑🏻‍💻 Enter a human software engineer

I went a couple of rounds with this flow before realizing that Copilot is not able to figure out the real issue. I visited the ReqRes site and saw that they had introduced a new API key header with a rate limit of 100 requests per day. No wonder the tests were broken due to 401

I tested one method after adding the header and then provided this context to Copilot to do further refactoring for me.

Looks like the reason why ReqRes tests were failing were because the site has introduced an API key on their site. I requested one and added it to testRegisterSuccessful. Could you apply this change to other tests and run them to verify the 1st issue is resolved and then focus on the 2nd part of the problem wherein report portal API returns us a 4XX error

I continued with the cycle multiple times, and it took some time for Agent mode to correctly refactor all the tests to add the new header.

But … ✋

The core issue of being able to pull a stack trace for a failing test remained.

Agent mode was not able to figure out the correct API sequence to fetch stack traces for failed tests.

I wasted close to 40-45 minutes trying to get it to “vibe code” the solution out as the cool kids suggest, after which I decided to explore the API docs for ReportPortal myself.

It turned out Copilot was close but not all the way, it was hallucinating some query params for the API that is able to fetch the failed results. I explored the Report portal UI and then, using Chrome developer tools, analyzed XHR requests to figure out the correct APIs that the application uses to display failed tests and their stack traces.

I then played around with the APIs in Postman and identified a sequence of steps that should work, and wrote it in a markdown file to provide as context

Agent mode was now able to modify the ReportPortalClient inline with provided APIs while also being able to figure out why the other tests were failing (due to API key being set in query params instead of header 🤦).

You can find the changes in this commit

⛔ Build failure again

Copilot modified a method in ReportPortalClient but did not update all the code references, leading to build failures.

Also, I noticed it kept on running all the tests in the workspace when we could only run one sample test to validate the flow.

I prompted it to only run the one case, and it did understand that the build was failing. However, it went on to make a bunch of nonsensical changes, leading to unimported packages, etc, again.

At this point, I realised. It was time to take a pause and jump into this problem and solve it the right way.

I fired up my trusty IntelliJ and started digging into the problem one by one.

I was able to fix the build failure quite easily and realised an earlier implementation was not required since we were already preparing a list of failed test items.

Below is the commit log and the commits that I made to fix the build problems. It’s funny, but as a developer, coding the solution yourself helps you understand it deeply and grasp its structure much better. When you are reviewing code, you tend to prefer speed over quality at each step, and also do not have a good mental model of classes, methods, and their relationship

- Update AutoTestAnalyzer to get list of failed test items directly and not query JsonNode

- Fixed a bug wherein `/api/v1` was being added unnecessarily

- Fixed ReportPortalClient urls

- Added a failing tag to run a targeted test

Commit 1

- Add model classes for Report Portal test item response and refactor AutoTestAnalyzer to utilize new structure

- Pull stack trace from failing tests description itself

- Updated loggers to log requests

Commit 2

I ran the failing test and saw that I could finish the loop till the point where I was able to identify a failing test, and then the analyzer returned a result

May 04, 2025 9:56:00 PM io.automationhacks.testinfra.TestAnalysisListener onExecutionFinish

INFO: Test execution finished. Starting test analysis...

May 04, 2025 9:56:00 PM io.automationhacks.testinfra.AutoTestAnalyzer analyzeFailingTests

INFO: Fetching failing tests from Report Portal...

May 04, 2025 9:56:00 PM io.automationhacks.testinfra.AutoTestAnalyzer analyzeFailingTests

INFO: Found 1 failing tests. Analyzing stack traces for suggestions...

May 04, 2025 9:56:00 PM io.automationhacks.testinfra.AutoTestAnalyzer logAnalysisResult

INFO: Test Item ID: 884

Suggestion: Assertion failed: 1 expectation failed.. Review the assertion conditions.

LLM-based test analyzer

At this point, I felt confident that the flow would work to hand back control to Agent mode.

I have made changes in ReportPortalClient and AutoTestAnalyzer to be able to pull the error stack trace. I want you to now focus on StackTraceAnalyzer such that it can detect a failure pattern and then provide a suggestion via an LLM or an Agent mode

I could now see that Agent mode smartly did manual analysis on a failing test and then added a fallback using an LLM call to OpenAI GPT 3.5. It also went ahead and added tests for the analyzer.

To be honest, this feature seems quite close to completion, and if I were to continue down the path. I feel like this could be added as an operational feature in ReportPortal for teams, of course, after the code is cleaned up and polished.

You could follow all the commits here on GitHub

Conclusion

I hope this practical example gives you a sense of what working with GitHub Copilot Agent mode looks like in actual practice

Here are my top key takeaways

1. Powerful: Agent mode is a powerful coding tool and can write code pretty fast for more complex problems and leverage tools (terminal, code search, external MCP servers) to write powerful solutions

2. Verify: It sometimes goes down wrong paths and makes nonsensical changes. As a developer, you should take your time and review each change. Don’t vibe code further if you don’t understand what’s going on

3. Learn: Knowing programming languages, frameworks deeply would help to separate good code from bad. Building taste requires time and practice.

4. Write code yourself: Sometimes, it's okay to take on the driver mode, pause the AI Agent, and write the code yourself. It may be faster and easier for you to maintain the code later on

5. Quality: Code quality produced by Agent mode is more like an eager early-career engineer. It lacks polish, but that could be fixed by adding in some Senior engineer intuition and more descriptive prompts

6. Not omnipotent: AI is not god, it does not know what it does not know. You have to hold it a little bit and provide the right context, prompts, and knowledge to achieve its tasks

7. Testing: Tests tend to be more unit test-like; you should consider writing functional tests for the code it produces as well

Overall, it's an impressive new shiny tool in your toolkit as a professional Software engineer, and you should use it, but with caution ⚠️ Mix it with your experience and intuition, and you can achieve impressive things.

References

1. Vibe coding with GitHub Copilot: Agent mode and MCP support rolling out to all VS Code users - The GitHub Blog

2. Model Context Protocol

3. Use MCP servers in VS Code (Preview)

Enjoyed reading this?

Subscribe to the newsletter and YouTube channel (@automationhacks) for more ⚡insights into software testing and automation, or connect on topmate for a personalized 1:1 session.

Disclosure: I’ll earn a small commission if you decide to purchase some of the educative.io text courses linked in the blog for your learning and growth. Their system design courses and grokking coding interview courses are arguably quite helpful for interview prep.

Until next time, happy testing!

Why use local LLMs?

You can publicly access LLM (Large language models) for your day-to-day workflows via any of the providers by visiting their site.

In this blog series so far on Gen AI for testing, we’ve seen how ChatGPT.com could be used to generate test cases by understanding prompting, Github copilot for chat, and Copilot edits can be used with available models from Open AI and anthropic. If you are a new reader; welcome 👋 and feel free to subscribe and check out previous editions on automation hacks

These models are wonderful and powerful on the web, however, for any sophisticated use case, you will need access to their API. It's quite cheap as of now but you’ll still need to get subscriptions ranging from $20 to $200 per month for OpenAI or Anthropic

With a business license, You mostly have guarantees that your data should be protected and secured but you may always have concerns at the back of your mind that your proprietary and confidential data is leveraged to train subsequent models, maybe even exposing your confidential company to data leaks. 🤷Plus a closed ecosystem blackbox is not a lot of fun, is it?

On the flip side, there is a thriving ecosystem of open-weight models that can be run on your local laptop or company infrastructure and leveraged for tasks suitable for LLMs. In this blog, we’ll take a look at Ollama which is one such solution built by the OSS community.

Ollama

I recently came across ollama, a wonderful tool that enables you to host local models and run inference. It has an MIT license code and 129K stars as of Feb 2025. Let’s understand how to set up ollama to host Deepseek and Llama, open-source models

Running LLMs locally and using CLI

We will run ollama using Docker to provide isolation and easy management of Ollama Docker image

Let’s download the ollama image from the docker hub and start the container by running below

docker run -d -v ollama:/root/.ollama -p 11434:11434 --name ollama ollama/ollama

Once this command finishes execution, we can verify that ollama is running by running below

docker ps -a | grep -i "ollama"

Next, let’s run a small model from meta LLama 3.2 with 1 billion params

docker exec -it ollama ollama run llama3.2:1b

Let’s also download deepseek-r1

docker exec -it ollama ollama run deepseek-r1

Other available models are

• Llama3.3 → 70B → 43 GB

• Llama3.2 → 3B → 2 GB

• Deepseek-r1 → 7B → 4.7 GB

Ollama also recommends certain heuristics for how much computing power you need to run these models locally

• 8 GB RAM to run 7B model

• 16 GB RAM to run 13B model

• 32 GB RAM to run 33B model

I have an Apple M1 Pro with 10 CPU, and 32 GB RAM and this should be able to run llama3.2 and deepseek-r1 models comfortably, however since RAM may be required for other system processes, we’ll make sure our docker resource access is limited

If you have docker desktop installed, you can go to Settings → Resources → Advanced and restrict the resources appropriately. For example

Once you run the above command, you should get a command line interface like below

You can type a prompt and get a response by using prompting techniques like we learned earlier.

Side note: Prompting an LLM is a super important skill to learn in 2025 and if you want to learn more you can refer to this cool course “All You Need to Know About Prompt Engineering” on educative to deep dive.

I’ll give the same prompt we gave earlier to generate some tests.

>>> """
... # MISSION
...
... 1. Generate test cases to test google homepage
... 2. Please Cover functional, non functional cases from web, mobile and backend perspective.
...
... # INPUT
...
... 1. Google home page is an app that takes the following inputs:
...
... * `Search box`
... *  User can either click on `Google search button` or `I'm feeling lucky button`
...
... # RESPONSE FORMAT
...
... 1. csv with columns as `Test ID`, `Title`, `Preconditions`, `Steps`, `Expected Result`
... """

You can see CPU and memory spike up as the model tries to generate text as per your requirements.

It may be a bit slow but given more computing (CPU and memory) it should be able to return results faster

Below is what deepseek comes up with, along with the familiar chain of thought being exposed at the beginning

Okay, so I need to create test cases for Google's homepage based on the user's requirements. Let me break it down. First, the user mentioned covering both functional and non-functional tests from web, mobile, and backend perspectives. The inputs are the search box and two buttons: Google Search and "I'm Feeling Lucky."

Starting with functional tests:

For the web interface:

Test if the search box works when you type a query and press enter.
Check if the search box updates the page correctly.
Verify that clicking either button takes you to the corresponding search result.

Now, non-functional tests for the web might include performance metrics like load time, response time, and error handling on invalid inputs.

For mobile:

Ensure the search box is functional but requires a long press or swipe for buttons.
Check app store ratings and download stats.

Backend-wise, I need to test if Google can handle multiple requests simultaneously without crashing.

Non-functional tests here would involve load times with many users and stress testing under high traffic.

Wait, did I miss any cases? Maybe performance on different devices or network conditions for mobile. Also, error handling when there's no internet connection—like not being able to sign in or search properly. 

Here is a structured approach based on your requirements:
Test Cases Overview:
Functional Tests:

Search Box Interaction: Verify the search box updates and submit results.
Button Clicks: Ensure both buttons trigger correct actions.

Non-Functional Tests:

Performance Metrics (Web): Measure load/response times; handle invalid inputs.
Mobile Functionality: Check touch interactions for buttons, and app store stats.
Backend Stress Testing: Evaluate response under high load.

Cross-Device Testing: Ensure functionality across different mobile devices and screen sizes.
Test Cases Table:
| Test ID | Title                                      | Preconditions                          | Steps | Expected Result | |---------|--------------------------------------------|----------------------------------------|--------||---------|--------------------------------------------|----------------------------------------|----------------------------------------------------------------------|------------------| | FT01    | Search Box Works                           | Open Google homepage                    | Type "example.com" in search box, press Enter                     | Search result displayed at top | | FT02    | Search Box Updates Correctly                | Open Google homepage                    | Type "example.com" in search box (slide down or up)               | Webpage updates with results            | | FT03    | Google Search Button Clicks to Home Page     | Open Google homepage, click Search      | Click on "I'm Feeling Lucky" link                             | Takes user to featured articles | | NFT01   | Mobile Search Box Functionality             | Open Google homepage (mobile version)   | Swipe up or long press for search box inputs                    | Results displayed correctly | | NFT02   | Mobile Google Buttons Clicks               | Open Google on mobile device            | Click "I'm Feeling Lucky" on mobile                            | Takes user to featured articles          | | NFT03   | App Store Metrics                           | Open Google homepage                   | Check app store ratings, downloads, etc.                      | Correct stats displayed                | | BFT01   | Backend High Load Stress Testing            | Simulate high traffic scenario         | Test multiple simultaneous searches                            | System handles load without crashing    |
Notes:
Ensure testing across different devices and screen sizes for mobile functionality.
Validate app store availability across platforms.

This covers both functional and non-functional aspects from web, mobile, and backend perspectives.

Not bad at all, eh?

How about a UI?

CLI is cool for a programmer but you may be used to having a web GUI at your disposal to see past chats and use UI to write your prompts and also provide additional details.

Ollama comes with a wonderful GUI that can be used for this purpose

Let’s set up open-webui by running below command

docker run -d -p 3000:8080 --add-host=host.docker.internal:host-gateway -v open-webui:/app/backend/data --name open-webui --restart always ghcr.io/open-webui/open-webui:main

Once you hit

http://localhost:3000/, it will bring up a familiar-looking web UI like below

This provides:

1. Access to the previous 30 days of chat

2. Model selector at the top

3. Chat interface to prompt LLMs

I tried to prompt it to tell me a joke and discovered that the previous generation in progress slowed down dramatically.

Reflection: In the current state even with 4 CPU and 12 GB RAM, the generation is quite slow and sequential. Boy, these LLMs while wonderful at text generation are super resource-hungry. I wonder if a day will come when the most powerful models will run on my phone at a fraction of the cost.

Also, deepseek-r1 by default uses reasoning and thus “thinks” longer while revealing its chain of thoughts (in this case 14 minutes for a joke). This example is trivial and not something to use the model for but still interesting to observe the reasoning patterns.

Does Ollama expose an API?

Now CLI and web GUI are cool and would help you with exploration in a more chat style, but what if you want to build some utility on top of LLMs, you gotta have an API

Reflection: 🧟God help you if you are trying to force fit LLMs to solve a problem that would otherwise be straightforward to solve via some other open-source library or tool, but hey, I don’t judge. You do whatever floats your boat.

Ollama has a well-defined API surface that allows you to:

1. Choose the model you want to use

2. Send a text prompt and get the answer in the desired format

For example, say I wanted to write a playwright test using Java to open Google search and type a keyword

I could invoke the below API /generate method with the below options:

1. Choose the model

2. Write my text prompt.

3. I also want to get all of the text in one shot so can set the stream as a false

4. I’ve set the temperature as 0.8 for it to be creative.

curl --location 'http://localhost:11434/api/generate' \
--header 'Content-Type: application/json' \
--data '{
  "model": "llama3.2:1b",
  "prompt":"Write a playwright test in java to open google search and type in keyword meta llama. Make sure the code is clean, concise and follows best practices for modularity",
  "stream": false,
  "options": {
    "temperature": 0.8
  }
}'

And it came up with below

{
   "model": "llama3.2:1b",
   "created_at": "2025-02-25T01:14:15.302218386Z",
   "response": "```java\nimport javax.swing.*;\nimport java.awt.*;\nimport java.awt.event.ActionEvent;\nimport java.awt.event.ActionListener;\n\npublic class SearchPanel {\n    private JFrame frame;\n    private JTextField inputField;\n    private JButton searchButton;\n\n    public SearchPanel() {\n        createAndShowGUI();\n    }\n\n    private void createAndShowGUI() {\n        // Create the main window\n        frame = new JFrame(\"Google Search\");\n        frame.setDefaultCloseOperation(JFrame.EXIT_ON_CLOSE);\n        frame.setLayout(new FlowLayout());\n\n        // Input field for user input\n        inputField = new JTextField(20);\n\n        // Button to trigger search action\n        searchButton = new JButton(\"Search\");\n\n        // Add input field and button to the frame\n        frame.add(inputField, new BorderLayout.CENTER);\n        frame.add(searchButton, BorderLayout.SOUTH);\n\n        // Connect search button's \"actionPerformed\" method\n        searchButton.addActionListener(new ActionListener() {\n            @Override\n            public void actionPerformed(ActionEvent e) {\n                String keyword = inputField.getText();\n                if (keyword.length() > 0) {\n                    try {\n                        Process process = Runtime.getRuntime().exec(\"gnome-terminal -e --title=Google Search -t google.com -m \\\"Search for \" + keyword + \"\\\"\");\n                        process.waitFor();\n                    } catch (IOException | InterruptedException ex) {\n                        JOptionPane.showMessageDialog(frame, \"Error searching for '\" + keyword + \"': \" + ex.getMessage());\n                    }\n                }\n            }\n        });\n    }\n\n    public static void main(String[] args) {\n        SwingUtilities.invokeLater(new Runnable() {\n            @Override\n            public void run() {\n                new SearchPanel();\n            }\n        });\n    }\n}\n```",
   "done": true,
   "done_reason": "stop",
   "context": [
       128006,
       9125,
       ...
       74694
   ],
   "total_duration": 21054774635,
   "load_duration": 15001166,
   "prompt_eval_count": 81,
   "prompt_eval_duration": 61000000,
   "eval_count": 339,
   "eval_duration": 20978000000
}

I printed this in a Python shell to get more readable text

python3
>>> print(""" <insert response here> """)

And it returned below.

```java
import javax.swing.*;
import java.awt.*;
import java.awt.event.ActionEvent;
import java.awt.event.ActionListener;


public class SearchPanel {
    private JFrame frame;
    private JTextField inputField;
    private JButton searchButton;


    public SearchPanel() {
        createAndShowGUI();
    }


    private void createAndShowGUI() {
        // Create the main window
        frame = new JFrame("Google Search");
        frame.setDefaultCloseOperation(JFrame.EXIT_ON_CLOSE);
        frame.setLayout(new FlowLayout());


        // Input field for user input
        inputField = new JTextField(20);


        // Button to trigger search action
        searchButton = new JButton("Search");


        // Add input field and button to the frame
        frame.add(inputField, new BorderLayout.CENTER);
        frame.add(searchButton, BorderLayout.SOUTH);


        // Connect search button's "actionPerformed" method
        searchButton.addActionListener(new ActionListener() {
            @Override
            public void actionPerformed(ActionEvent e) {
                String keyword = inputField.getText();
                if (keyword.length() > 0) {
                    try {
                        Process process = Runtime.getRuntime().exec("gnome-terminal -e --title=Google Search -t google.com -m \"Search for " + keyword + "\"");
                        process.waitFor();
                    } catch (IOException | InterruptedException ex) {
                        JOptionPane.showMessageDialog(frame, "Error searching for '" + keyword + "': " + ex.getMessage());
                    }
                }
            }
        });
    }


    public static void main(String[] args) {
        SwingUtilities.invokeLater(new Runnable() {
            @Override
            public void run() {
                new SearchPanel();
            }
        });
    }
}
```

I don’t know Playwright API well enough yet but this probably looks wrong, for some reason Llama thought it needed to create a Java AWT component for search

In reality, it should look something like below

package org.example;


import com.microsoft.playwright.*;


public class App {
    public static void main(String[] args) {
        try (Playwright playwright = Playwright.create()) {
            Browser browser = playwright.chromium().launch();
            Page page = browser.newPage();
            page.navigate("http://playwright.dev");
            System.out.println(page.title());
        }
    }
}

Friendly reminder: models could be wrong and we as engineers need to use judgment to choose the right one for the use case and prompt them well, I think providing some examples of what you are looking for may help yield better results but that’s an exercise I’ll leave to you.

And that’s all folks! 🥕

Conclusion

Let’s do a quick recap, shall we?

Hopefully, you take away the below

1. Ollama can run on docker containers and run open weights models on your infrastructure like Llama, deepseek, etc

2. You can use CLI, Open Web UI for chat, or an extensible API to build on top of

3. Choosing the right model/size for the use case is important

4. Deepseek-r1 reasoning models are slower but can spew more accurate results

5. Models can be wrong or hallucinate; don’t forget to check their work and learn the topic yourself before using their output.

The world still needs you 🫵, you human engineer. Don’t gaslight yourself with Gen AI hype and give up just yet! I’m rooting for you!

Enjoyed reading this?

Subscribe to the newsletter and YouTube channel (@automationhacks) for more ⚡insights into software testing and automation, or connect on topmate for a personalized 1:1 session.

Disclosure: I’ll earn a small commission if you decide to purchase some of the educative.io text courses linked in the blog for your learning and growth. Their system design courses and grokking coding interview courses are arguably quite helpful for interview prep.

Until next time, happy testing!

Andrej Karpathy probably needs no introduction, but if you are hearing the name for the first time, he is a world-renowned expert in AI and deep learning who knows his stuff having led major projects at Tesla, OpenAI

This video shines a light on how LLMs are trained, what they are good for, and what they are not and I found it highly practical. It does not explain math, or underlying algorithms but gives you a solid intuition about the current landscape

What I took away from this?

• Models are extremely good for brainstorming generating ideas, and writing code but by definition act like stochastic parrots often imitating data provided by a human expert and generate

• We should not blindly trust what it generates or deploy it in mission-critical areas as it will have hallucinations. While we should use it for inspiration, we should *always review and verify its outputs before* using it for production-grade work. They are wonderful tools in our toolbelt and we should leverage them to make ourselves more productive.

What are the major steps?

Pre-training: LLMs are first trained on WWW text to form a base models where text is converted in tokens. A model can have X parameters which are knobs that can be tuned to get different outputs and are the vector of probabilities that LLMs use to produce outputs.

RLHF (reinforcement learning from human feedback): Human labelers sit down and write ideal responses to questions which the model then tries to imitate. This leads to GPT models like GPT4o. They work extremely well in verifiable domains like math or code

RL (reinforcement learning) is the next evolution where model outputs are scored by humans and a reward model to nudge the model into producing relevant outputs. Reasoning models can produce their chain of thoughts where they check their work, and backtrack if required to improve accuracy like Open AI o1, o3. This approach helps models perform better in unverifiable domains

Tips

You can ask models to use web search, or code to perform better at tasks like math

Tools

• Tiktokenizer: convert human-readable text into tokens that the model uses

• Hyperbolic: cloud to run models

• Together AI: cloud to run models

• Hugging Face: download data sets

• LM Studio: Download local models

Stay up to date with news

• AiNews.com

Highly recommend watching the video

Deep Dive into LLMs like ChatGPT

and thank you Andrej for putting this out in the world.

#AI #GenAI #LLM

Introduction

Github Copilot recently introduced the ability to edit multiple files in VS code. As of writing, this feature is currently in preview and titled Edit with Copilot

The possibilities with this capability are immense. You can now pair more efficiently with Copilot and ask it to make changes in your codebase across multiple files in natural language

I took this feature for a spin on the test-infra codebase with a straightforward use case and found the results surprisingly accurate.

In this blog, Let’s take a look at an example workflow step by step and understand how to work with this feature and then also discuss some ideas on how this could be useful in your day-to-day workflows

I use Jetbrains IntelliJ IDEA IDE for most of my professional development, but this capability is not exposed yet on the GitHub plugin there. We’ll use VS code to see this workflow. If you are not familiar with VS code yet you can read this to understand some of the basics

Without further adieu let's dive in 🏊

Launch Edit with Copilot

When you start Copilot edits in VS code you are greeted with this UI. You can add files to your working set. In this case, I have a test file opened which was added by default and we can also add additional files

Another useful feature is that you can switch the model that would be used for code generation under the hood such as Claude 3.5 Sonnet from Anthropic or GPT 4o from Open AI

Describe the change

Copilot edit is more like an experienced pair engineer in the driver role while you adopt the navigator/observer/reviewer role and instruct to perform some edits.

To begin with, You should have a clear idea of what change you want to make in the codebase. Quite often this is the most challenging aspect of working on large codebases because it implies you have a solid understanding of the code structure and what’s happening under the hood.

If you are not sure where the changes need to be made, the results probably won’t be great.

In such cases using chat to figure out higher-level solutions or brainstorming before you jump into edits could be a good way of thinking about this.

Let’s say I have the below test method ReqResCreateUserTest and I want to make a simple change to introduce a BaseTest class and move the setup method to that. There are other test classes in the same package that have this duplicated code that I would want to apply this refactoring later on

package io.automationhacks.testinfra.reqres.users;

…

public class ReqResCreateUserTest {

    @BeforeClass(alwaysRun = true)
    public void setup() {
        RestAssured.baseURI = "https://reqres.in/api";
    }

    @Test(groups = {Team.IDENTITY, Groups.SMOKE})
    @Service(Services.CREATE_USER)
    @Attributes(attributes = {@Attribute(key = "team", value = "identity")})
    public void testCreate() {
        String requestBody = "{\"name\": \"morpheus\", \"job\": \"leader\"}";

        given().contentType(ContentType.JSON)
                .body(requestBody)
                .when()
                .post("/users")
                .then()
                // TODO: Broken test example, change to 201 to fix
                .statusCode(200)
                .body("name", equalTo("morpheus"))
                .body("job", equalTo("leader"))
                .body("id", notNullValue())
                .body("createdAt", notNullValue());
    }
}

I prompted something like the below

Help me create a base test class in testng that all the classes in package io.automationhacks.testinfra.reqres.users; inherit from and that has the common setup method to set the base URL

And copilot edit understood the 2 main changes required for this

1. Create a new base test class

2. Modify the existing class to inherit

It created the base class pretty well

It was also able to make the change in inheritance hierarchy for the test file

Accept or discard changes

I also got an option to choose if I want to Accept or Discard this change and an option to cycle through all the changes

I also realized that I was not explicit that in this refactoring, I wanted copilot to also remove the setup method from the test classes so I made a follow-up prompt

Remove the setup method in child classes

And Copilot was quick to grasp that, it also enhanced the prompt with more context like “since it is already inherited from the base class” 🤔

And made the change as expected

I realized that there are a few unused imports left in the file and while it’s easy to make this change yourself, I figured why not let Copilot do this job. 🤷

Also make sure to remove any unused imports

And I could see the imports optimised

Like a responsible engineer, I ran the test to make sure it still works with this change and it was fine.

Note: The test here fails due to legitimate reasons. I had changed the expectation of the assertion on purpose to check some test infra components that I’m building when a test fails.

Reflection: This entire workflow took couple of minutes or so and was quite fast. I did notice that you have to constantly adopt the hat of a reviewer and ensure that the change Copilot edits make is in line with your expectations. It’s important to keep the batch size small so that you can reason about the changes.

Maybe this is what future engineers would spend most of their time on. Read, understand,d and review code over and over. I would imagine this could be quite taxing, in the long run, to constantly read lots of code but don’t have anecdotes on long-term use yet

Change in multiple files

I became more excited and thought why not apply the same change to all the files in the user package and prompted below:

Please apply the same changes in all files in users package

Relative path: src/test/java/io/automationhacks/testinfra/reqres/users

Copilot went ahead and applied the change but also made some additional changes like changing groups or even the expected status code.

🤯 It also figured out from the comments that 204 was the expected status code and I had introduced some logic to make the test flaky and corrected that, although with wrong syntax.

So, I went back and gave a more clear expectation

Please make below changes in all files in users package

Relative path: src/test/java/io/automationhacks/testinfra/reqres/users

1. Ensure the class extends BaseTest

2. Remove any unused imports

3. Do not change the @Test annotation or anything in the test body

Copilot did the right job and correctly updated all the files within the user's package. 🎊

Reflection: Copilot is not a mind reader, while it has a lot of context and can try to infer some things, you still need to be concise and clear on what you want it to do. Knowing how to prompt well and good English writing skills is a good skill to have in 2025

Ideas

This feature is exciting, it expands the use case of code generation and assistance into more of an editing workflow. This has the potential to make each developer significantly faster in repeating changes across a codebase or improving it methodically

I can see some obvious use cases for this

1. Given file X as an example, write a test that does Y

2. Implement a new function in existing library code that does Z and then write tests for it

Closing thoughts

After this pair programming session, I realized below

This could be a future mode for software development and is a natural evolution of IDE’s capabilities. It brings more intelligence to the code-editing process

Yes, if we are being honest. This was not at all a complex change.

Even simple IntelliJ refactorings can achieve this change pretty fast with auto code formatting in place or even a human hand coding this.

But the idea here was the explain the feature in a simple-to-understand workflow. The capability and the accuracy of edits are quite amazing.

This could be a really useful addition to a programmer's toolbelt and you could use it for increasingly complex tasks. God knows we need all the help we can get to manage the increasingly complex software engineering space.

Also, this reinforces that as a Senior engineer, you cannot slack off, you need to understand the underlying concept you want Copilot edits to apply to your codebase.

I see a risk that human software engineers may lose a bit of muscle memory when it comes to hand coding so it's important to keep them sharp with doing some work yourself as well. But this has been the story of software engineering anyway. We are moving towards higher levels of abstraction and better tooling 😇

References

1. Copilot Edits

2. Exploring Generative AI

GitHub Copilot is a fantastic addition to the professional software engineers toolbelt and can help with lot of common day-to-day tasks. I’ve been leveraging its capabilities heavily over the past year and I felt it makes a ton of sense to unpack this in a bit more detail.

In this blog, we’ll use GitHub Copilot to illustrate these interaction modes

1. Autocomplete suggestions as you type code in your IDE (Integrated development environment)

2. Inline chat for a specific piece of code or problem

3. Chat with commands and references

If you are using JetBrains IDE’s like IntelliJ or Pycharm or Visual Studio code

1. Please make sure to install the Github Copilot plugin or extension in your IDE of choice

2. Then, follow the steps to authorize them to access your GitHub account.

GitHub recently made Copilot free up to a limited set of autocomplete or chat interactions 🤩 and you can use this to play around with its capabilities 🙌.

Autocomplete ✅

I truly love this feature. ❤️

Why?

So much of writing automated tests involves writing similar code leveraging existing classes, utilities, or helper functions and assertions.

GitHub Copilot can help by helping you speed up the test authoring aspect. It autocompletes code for you as you type by displaying the suggested code in grey.

If the suggestion looks acceptable, you can press the TAB key, and that code is inserted at a given caret location. You can then edit the code to suit your specific requirements. You could also partially accept a suggestion as well.

If not, you can continue writing code and it will try to figure out what you are trying to write and suggest.

Could you make it more effective?

Yes, there are a few distinct things you can try to help Github Copilot make better suggestions. Let’s go over them next

Open relevant files

GitHub Copilot gives preference to what files you have open at the time that you are trying to write a piece of code. It also has context of your workspace but providing targeted reference to correct files would increase the chances of getting a more relevant autocompletion.

This makes logical sense since you would probably not have unnecessary files open when working on a codebase anyway.

Here are a few common use cases

1. If you are trying to write a new test that follows a given pattern that is already repeated in another file, having that file open would help Copilot suggest better autocompletes.

2. If you are writing assertions on a given test method and want to reuse existing methods then having the assertions file open would allow Copilot to suggest relevant suggestions for better

Write only the method signature

You can write the method's signature with its name, parameters, and types, and the GitHub copilot will auto-complete that function in your IDE.

This works better in strongly typed languages like Java, but you can also use Python-type hints to provide sufficient context to Copilot

Let’s take an example

Say you want to write the below function that returns epoch seconds for the current UTC time in Java

If you write below:

public long getEpochSecondsForCurrentTimeInUTC() {}

You can see Copilot takes the function signature and suggests an implementation

public long getEpochSecondsForCurrentTimeInUTC() {

return System.currentTimeMillis() / 1000;

Write an inline comment

Another way of getting autocompletes is to write a short comment stating what code you want to generate and Copilot takes that as input and generates code along those lines

For example below, I’ve prompted Copilot to generate a function that gives me a delta date by describing what I’m looking for

// Write a Java function that takes a date and time format and no of days and can return the date after adding the no of days

// Write a java function that takes a date and time format and no of days and can return the date after adding the no of days

public String addDaysToCurrentDate(String dateFormat, int days) {

SimpleDateFormat sdf = new SimpleDateFormat(dateFormat);

Calendar c = Calendar.getInstance();

c.add(Calendar.DATE, days);

return sdf.format(c.getTime());

}

Generate tests 🧪

What if you want to test your utilities?

Say you want to test the previously written test utility method, you can put the caret at the relevant position and then Right Click → select GitHub Copilot → Generate tests

This will initiate a Github Copilot chat session and generate tests for this simple function.

You can then copy and paste these methods and put them in a test file to have some instant coverage. Also notice, it uses the file DateTimeHelpers file as a reference

The same technique could be used to generate unit tests for any simple function. If the function you are testing is a bit complex, then you may have to do some back-and-forth prompting to arrive at a good set of tests.

Another tip is to have some example tests whose style you want to mimic.

Having that file open may result in better results, you can also prompt stating please generate tests following the coding style followed in <X.java> and it would consider them giving you more accurate tests following your project’s style guide.

You can also see the generated tests here

Inline chat 💬

Inline chat is a way to prompt Copilot to do something in the context of the code you are working with.

Continuing our previous example, say I autocompleted getEpochSecondsForCurrentTimeInUTC method and I want to write a comment explaining the method.

Clean code Tip: Your code comments should always explain the why or what and not the how. Your code should be clear and readable so that the reader can understand the how by just reading the code

You can press Opt + enter (on Mac) and then select Open GitHub Copilot Inline chat

Now type in your prompt

in this case, I just prompt it to generate a comment

You can now see the suggested comment and even a side-by-side diff if required

Side by side view can help you contextualize where the change would be inserted

Now this is just one example, you can use this to get Copilot to do specific generation in the context of your current code. For instance,

1. You could use this way to prompt and generate some specific code

2. or tests for a given condition.

3. Fix a problem with the code

Copilot chat 🗨️

Beyond autocomplete and inline chat interaction modes that we saw above.

You can also start a regular chat session to ask Github Copilot to do something for you

This is similar to many other chat interfaces and you can leverage the same prompting techniques that we discussed in the 1st blog of this series.

Gen AI and Test Design: Supercharge your Prompting skills to get better results.

Add specific references

A couple of interesting new features here are:

You can choose which specific file you want the chat to take as a reference by clicking on the plus

Follow-up prompt suggestion

And that Copilot also provides you with a suggested next message

Use the forward slash to see commands

Unlike other LLMs, Copilot is tailored for coding and as such provides you with convenient commands to use. You can press forward slash and then review available actions.

This is a rapid way of prompting Copilot to perform these common operations.

You can select a piece of code and then quickly use /doc /simplify /tests /fix to prompt Copilot to take action on them

For E.g. below I just asked it to generate a doc for the newly added method and Copilot suggested a pretty nice comment for me.

Recap

Let’s recap what we learned in this blog

1. Github Copilot offers coding assistance with autocomplete, inline chat, and chat interfaces

2. Keeping relevant files open or explicitly proving context helps get better results

3. You can provide a method signature or inline comment to prompt Copilot to take action

4. Inline chat helps you get contextual changes at the caret position

5. You can leverage chat commands to perform common actions like fix, explain, generate tests, simplify, generate docs, etc. I’ll leave it to you to try these on your own

In the next edition, we will explore Github Copilot multi-file edits

Subscribe to the newsletter and YouTube channel (@automationhacks) for more ⚡insights into software testing and automation, or connect on topmate for a personalized 1:1 session.

Learn the GenAI landscape from a user perspective and prompting tips to generate a test design for the Google Home page

Overview

It’s 2025, and at this point, GenAI does not need an introduction. November 30, 2022, shook the world when Open AI launched ChatGPT and captured the imagination of people worldwide.

We live in a world now where we can ask GenAI to solve different kinds of problems for us like

1. Generate code

2. Edit multiple files

3. Generate images, videos

4. Ask questions about problems and incrementally build solutions

It’s fascinating to see this technological progress, even though there seems to be a lot of noise and hype around it.

Many people are putting out clear signals. I feel like we all got a cool new technology upgrade added to our toolchains as a professional software engineer, and we should leverage it in the right way

I’ve so far been silently watching this space evolve. I use many of the tools and techniques in my daily workflows, and I feel it's time to put my take on how AI is helpful for testing.

I’m by no means an expert but I’ll build and learn in public and hopefully take you with me on this learning journey.

My current mental model 🧠

Let’s adopt the POV (point of view) of a senior engineer focussed more on the user layer and how can they leverage and integrate GenAI in their day-to-day workflows

Side note: Stephen Wolfram wrote a fascinating piece on What Is ChatGPT Doing … and Why Does It Work? I recommend reading this if you want a better intuition on how this technology really works under the hood.

Interfaces 📱

I see 5 main ways to interact with GenAI and get it to do something helpful for you

1. Chat: Type a question in a chat box and see a streaming answer being typed back

2. Inline chat: While coding you can type a prompt in the context of the code you are working on

3. Voice: Speak your prompt out loud in natural language and hear the response back

4. Autocomplete while coding: Coding assistants predict what you want to write and then provide an autocomplete suggestion

5. Multi-file editing: Coding assistants provide context on a few files where you want to make a change and then type a prompt for GenAI to make that change for you.

Input modes ➡️

We can write prompts or type in your question or task along with the required context such as:

1. Ask a question and follow up

2. Prompt to make an edit and accept/discard

Outputs ⬅️

And get back a result in the below formats that you can then either use as is or refine further in more incremental cycles of back and forths

1. Text

2. Code generation

3. Transform text into images or videos

Depending on your use case you may use one or more of these interfaces to get to the desired outcomes.

Vendors

Who are the major vendors at the moment?

There are few players who have managed to corner a large section of the customer base at this moment and more keep on coming every few weeks.

⚠️Caution: You may be tempted to try to keep up with every single change in AI and while you can certainly try, it can quickly turn out to be quite exhausting. I’ll suggest sticking to your core tools in the beginning and being comfortable with them first and taking one new tool to tinker and experiment with as you form a better intuition for yourself. If you find it better for your workflow, please switch by all means, But there should be a method to madness isn't there?

Here I’m focussing on solutions that are relevant to primarily coding and testing.

General purpose

1. ChatGPT

2. Gemini

3. Microsoft copilot

Search

1. Perplexity

Coding

1. Github Copilot

2. Claude

3. ChatGPT

IDE

1. Cursor

2. VSCode

3. JetBrains IDE

Sometimes vendors are evolving more into a platform where they provide a layer of abstraction over different models and allow the user to switch between them like VSCode with Github copilot supports both OpenAI and Anthropic’s models

Major models

In no particular order

1. OpenAI

2. Claude

3. Google Gemini

4. Llama

Each of these models provides sub-models of different sizes like small, medium, and large.

What size is useful for what task is still not an exact science at the moment but as we use it more, we’ll probably form a better intuition and have more research further down the line

With this solid foundation of what technological surface areas we are working with.

Let’s now dive into each and see how it could help us with testing, shall we? 😀

Generate test cases with LLMs

Before you explore anything else, learning how to chat well with LLMs is super important as you’ll see. This forms the basis for using chat interfaces and inline chat well during problem-solving and coding.

When it comes to chatting efficiently with LLM, we need to keep in mind below

1. Specify the outcomes you want LLM to produce.

2. Learn how to provide the necessary context to the model

3. Provide examples and clear steps or instructions

4. Ask it to perform a chain of thought by adding think step-by-step

5. Specify the role/character you want the LLM to adopt

Let’s understand these with some examples.

You can find these prompts and generated output in this GitHub repo

Clear, detailed, and specific prompts

LLMs are great at parsing prompts written in markdown format.

Below is an example format suggested by a few testers wherein you specify a goal/mission/task upfront, provide context in the input section, and specify what response format you want the answer in

For e.g. csv format for test cases or JSON for test data that you can feed into your automated tests.

Below is a canonical test plan for the Google search page.

🙄Yeah, I know, you would have seen this ton of times already but the beauty lies in its widespread availability and simplicity. We all know intuitively how to test the Google homepage, don’t we?

# MISSION


1. Generate test cases to test Google homepage
2. Please Cover functional, and non-functional cases from web, mobile, and backend perspectives.


# INPUT


1. Google home page is an app that takes the following inputs:


* `Search box`
*  User can either click on the `Google search button` or the `I'm feeling lucky button`


# RESPONSE FORMAT


1. csv with columns as `Test ID`, `Title`, `Preconditions`, `Steps`, `Expected Result`

You can see LLM here does a pretty good job in providing a high-level test plan covering web, mobile, and backend tests. You can find the prompt and output here

With this context set, we can now go ahead and refine our prompts and then ask it to generate cases around a specific theme like focus on UI, usability, accessibility, or performance.

⚡Tip: You can also add a section in the prompt above-called steps and provide a numbered list of tasks or approaches you want the LLM to follow while coming up with the response as well

While this mode of formatting with markdown is quite precise, you can also chat in natural language.

LLM tools are now getting better day by day at ensuring your prompts are cleaned and enhanced before sending to the model.

As they improve further, even people not well versed with prompting would be able to get similar results at people who are quite creative, but there will still be some differences.

LLMs cannot read your minds so you need to provide all the details required to do the job well, just like you would mentor or coach an n00b engineer on your team. A good heuristic is, if you show your prompt to another human would they be able to infer what you are trying to get the LLM to do? If not, it is time to refine your prompt more

⚡Tip: If you are confused, you can also ask the model itself for suggestions.

For e.g. I asked for suggestions from ChatGPT on improving my prompting

How could I have improved this prompt itself in order to get better results and more test cases covering different aspects? Please explain it to me step by step.

Below is what ChatGPT came up with apart from a pretty long explanation of why this works.

Generate test cases for the Google homepage. Cover the following areas: functional, non-functional, security, accessibility, performance, and usability. Test the web (desktop and tablet), mobile (responsive and native), and backend services.

The homepage includes:

A search box with autocomplete.

'Google Search' and 'I'm Feeling Lucky' buttons.

Voice search (mobile-specific).

Test input variations like valid inputs, invalid inputs, edge cases, and security attempts. Non-functional aspects should include performance (load time), accessibility (WCAG 2.1 compliance), and compatibility (across devices and networks).

Output the test cases in CSV format with columns: Test ID, Title, Preconditions, Steps, Expected Result. Provide at least 5 test cases per type of testing

It added specific areas to focus on:

security, accessibility, performance, and usability.

And also which platforms to consider

Test the web (desktop and tablet), mobile (responsive and native), and backend services.

Then it also added aspects or specific areas where we could focus the LLM on

Test input variations like valid inputs, invalid inputs, edge cases, and security attempts. Non-functional aspects should include performance (load time), accessibility (WCAG 2.1 compliance), and compatibility (across devices and networks)

You can see the prompt and its output here

And below is what the model returned.

You can see with the above the test plan has more specific cases on security, and usability aspects that were previously missing.

In general, the below tips work well while prompting

1. Use bullet points or numbered lists to call out specific actions you want the LLM to do

2. Be clear and specific, cut out any fluff context that is not relevant

3. Use markdown and bold or italics to put emphasis if you want the LLM to focus more on a specific aspect.

4. Specify how you will use this data and the format in which you need the output.

You could read the anthropic prompt engineering guide for more examples

Be clear, direct, and detailed - Anthropic

Provide clear examples

Also referred to as multi-shot prompting

I liked this example from the anthropic guide wherein we can have a lot of control over the output by being detailed in our prompts and specifying some examples

We can follow the same example while trying to generate more targetted functional tests by using prompt

# MISSION


- Generate test cases to test google homepage
- Please cover the following areas
 - Functional cases for the web (desktop and tablet), mobile (responsive and native), and backend services.
 - Test input variations like valid inputs, invalid inputs, edge cases


# INPUT


1. Google home page is an app that takes the following inputs:


* `Search box`
*  User can either click on `Google search button` or `I'm feeling lucky button`


# STEPS


You can follow below examples while preparing the test case in csv format


- Test ID: FT-WEB-01
- Title: Test google search for valid search params
- Preconditions
 - Given google search page is loaded
- Steps:
 - When user types a valid search query in the search box
 - And clicks search button
- Expected Result
 - Then google search results page should show relevant results


# RESPONSE FORMAT


- CSV with columns as `Test ID`, `Title`, `Preconditions`, `Steps`, `Expected Result`
- If there are multiple steps ensure there is proper line break between them

This produces a slightly better-formatted test case focussing on functional cases. You can see the prompt and tests here

Chain of thought (COT)

For complex reasoning, you can increase accuracy by asking LLMs to explain their reasoning by adding the phrase “Think step by step”

This can be useful while you are learning about a topic or thinking of test cases.

You can leverage this to get an insight into how LLM arrived at a certain test case and also force them to be more rigorous, thereby reducing the error rate.

By this time, this chats context window already has some preloaded context that we are trying to design test cases for the Google homepage

Let’s use this technique to get LLM to explain its reasoning behind focussing on nonfunctional cases specifically on resiliency testing

Your task now is to focus exclusively on resilience testing for the backend

Please output the result in CSV format like before and also think step-by-step

LLM provides a good mental model around resilience testing

Understand Resilience Testing Goals:
Resilience tests aim to assess how a system behaves under abnormal or stressful conditions, ensuring it recovers gracefully or minimizes service disruption

Identify Key Failure Scenarios:

• Backend unavailability (e.g., server downtime).

• Network-related delays or high latency.

• Failures in Dependent Systems (e.g., database or autocomplete services).

• Sudden spikes in load or malformed requests.

Simulate Real-World Failures:

Use common failure scenarios such as API unavailability or service crashes.

Incorporate tests for handling unexpected input like malformed requests or excessive data loads.

Define Recovery and Graceful Degradation:
For each scenario, ensure the system can recover quickly or degrade functionality without crashing.

Specify Observations and Outcomes:

Each test includes clear conditions (e.g., simulated failure or delay).

Expected results focus on user-friendly error messages, stable service performance, or appropriate system logs.

And we get below cases which seem quite reasonable to me as a starting point to build on top of. You can read the prompt and see tests here

⚡Tip: What would have been better is to provide the cURL of the given search API and its downstream as additional context to get some targetted tests, but I’ll let you explore that on your own

Role-based prompting

This tip has been mentioned so many times that I’ll just drop a one-liner and you can see full prompt and tests here

Generally adding a line to instruct the role you want the LLM to play works out better as it helps reduce the solution space

You are a world-class experienced staff software development engineer in test at Google

By adding the above, I can observe the LLM suggested a few additional cases that probably an experienced staff engineer may think of.

1. Network partition scenarios: Focus on the system’s ability to maintain partial functionality.

2. Stale cache handling: Ensures resilience when dealing with outdated or corrupt cache entries.

3. Incorrect data from dependencies: Verifies robustness against unexpected or incorrect responses from dependent services.

New cases:

Test ID: RT-BACK-08
Title: Verify API response under network partition scenarios (New)
Preconditions: Simulate a network partition between the API and dependent services
Steps:

1. Disconnect the network to one or more dependent services.

2. Send a valid search request.
   Expected Result: The API continues to serve partial responses with a clear error or warning about degraded functionality.

Test ID: RT-BACK-09
Title: Verify API's handling of stale cache scenarios (New)
Preconditions: Introduce stale or corrupted cache data for API responses
Steps:

1. Trigger a cached query by sending a previously searched term.

2. Monitor if the API detects and refreshes stale cache data.
   Expected Result: The system refreshes or invalidates stale cache entries, ensuring accurate responses.

Test ID: RT-BACK-10
Title: Test response when dependent services return incorrect data (New)
Preconditions: Simulate incorrect or unexpected data from a dependent service (e.g., malformed suggestions).
Steps:

1. Send a search query dependent on the failing service.

2. Observe API behavior and logs.
   Expected Result: The API filters out invalid data, logging the anomaly while maintaining functional responses.

Summary

Hope this introduction was helpful

To summarise in this blog you learned

1. What is Gen AI?

2. What does the current landscape look like in terms of interfaces, input, outputs, vendors, and major models?

3. How to leverage Gen AI to design cases

4. Efficient prompting techniques such as being specific, providing examples, using markdown, using chain of thought, and role-based prompting

References

1. Exploring Generative AI

2. Prompt engineering overview - Anthropic

3. What Is ChatGPT Doing … and Why Does It Work?—Stephen Wolfram Writings

Coming soon!

In the next vol, we will focus on code assistance and inline chat with Github copilot. Subscribe to the newsletter to not miss out on that.

If you found this helpful, 🌱 Subscribe to my newsletter and YouTube channel (@automationhacks) for more ⚡insights into software testing and automation, or connect on topmate for a personalized 1:1 session.

Recently, a bright, up-and-coming SDET at work asked me:

How can I grow in my career?

How can I become a better engineer?

These are timeless questions. All high-growth engineers keep asking this often of themselves and their mentors or managers

It was wonderful to see them have self-awareness in their early career and the drive to improve themselves. I wish I had started asking about this earlier. 🙂

I’ve pondered on this question myself and often do self-reflection from time to time on this, and while I probably cannot give you a magic formula of success that would catapult you to the next level

I can share my take on it.

Disclaimer:

1. Please take it with a grain of salt 🧂 as this contains my unconscious biases.

2. When in doubt, do what feels is the best course of action for yourself in your unique life situation. There are no black and white answers here and anyone who says otherwise has something else on their agenda 😉

Phew! With that ominous disclaimer out of the way. Let’s dive in

Progression in your career realistically can be a combination of many factors -

1. sometimes it is a raw skill, talent, and execution

2. sometimes it’s the relationships you’ve built that work for you

3. often it's being at the right place at the right time and showing up with the right attitude

4. Most often it is luck or foresight to work on high-impact projects

5. a favorable sponsor, mentor, or manager

and what have you?

Surely you may be thinking, These do not seem very deterministic, do they?

Since there are only a few aspects that you have personal control over.

I’ll suggest we start from there.

🪜 Level Expectations

Before we look at skills that an SDET should master, let's set some broad behaviors and expectations that leaders reasonably expect from engineers at different stages of their career journey.

This is not an exhaustive list by any stretch, but it is meant to give you a ballpark idea and some high-level intuition

At each stage of your journey

You should assume the person has demonstrated performance indicators on the previous level and has the potential to reliably and consistently demonstrate at least 30% of next-level skills.

Entry/mid-level - A new hope; solid executor

Project impact:

• Strong execution on one team and ownership of shipping tasks on time

• Can design and execute test plans for independent features with guidance from senior engineers

• Can add incremental features to framework and tools for well-defined problem spaces and domains

Engineering excellence:

• Expert in one area (web, mobile, backend)

• Can code fluently in one programming language

• Has a solid understanding of testing fundamentals and techniques e.g. boundary value analysis, equivalence partitioning, decision tables, state transition tests mind mapping

People impact:

• Can communicate clearly with stakeholders and help drive bug fixes

Direction:

• Limited to a small/medium problem space

Senior - An independent, reliable, and seasoned engineer

Project impact

• Leads execution for a functional area in a group

• Deliver outcomes on medium complexity features with minimal or directional guidance

• Proactively takes ownership, designs, and builds scalable and efficient test frameworks and tools to solve testing problems

Engineering excellence

• Expert in more than one area (web, mobile or backend)

• A polyglot programmer and can code fluently in more than one language, with a good understanding of design patterns and approaches, refactoring techniques

• Understand sophisticated testing techniques and approaches like mocking, stubbing, integration testing, mutation, etc

• Good understanding of associated technology stacks: test runners, code coverage, CI/CD, reporting

People impact:

• Mentor early career engineers to grow into senior role

Direction:

• Influences technical direction for one area

• Effectively communicates technical concepts to both technical and non-technical audiences, and influences technical decisions.

Staff/Principal - A deep expert, who sets technical direction, large functional area influence, or industry impact

Project impact

• Leads execution of complex projects across multiple cross-functional teams

• Aligns testing efforts with broader organizational goals anticipates future challenges, and drives strategic initiatives.

• Able to deliver complex projects involving either deep technical skills or cross-team collaborations with no supervision and occasional directional feedback

• Able to build a robust, scalable, and comprehensive test strategy covering different app stacks

Engineering excellence

• Expert across domains (web, mobile, backend)

• Solid hands-on grasp on nonfunctional testing (resilience, chaos, load, security)

People Impact

• Motivated to mentor and teach senior engineers to grow into staff and guide peer managers on nuances in the technical domain

• Able to handle ambiguity and navigate organizational dynamics to drive progress

• Inspires and motivates others, builds high-performing teams, and fosters a culture of excellence

Direction

• Influences technical direction and identifies opportunities for innovation

• Leads the development and implementation of new testing methodologies and tools across teams and multiple areas

There are levels further than this in large companies like a Senior principal engineer, or distinguished engineer

Read more

Why?

Once you have a running instance of ReportPortal with one or more teams using it successfully, it becomes important to keep patching it with upgrades and ensuring you get the benefits of bug fixes and new features.

But, If not done carefully, you can risk losing data (launches, logs, and dashboards) and setting up integration again.

Trust me, You would not want to be the one who loses data for the company, especially when people depend on previous analysis, widgets, and dashboards for their day-to-day operations. 🤦

In this blog, I’ll help provide clear steps to achieve the below workflow:

1. Take a backup of Postgres db

2. Take a backup of binary data from Minio

3. Make changes in docker-compose.yml files (e.g. opening up ports, and adding new components like OpenSearch)

4. Ensure existing containers are updated.

5. Restore data into Postgres and binary

This blog is inspired by the guides published by the ReportPortal team.

I intend to explain nuances to someone new to Docker, Postgres, and Minio. If you are curious you can use them as additional resources

1. Backup & Restore Guide | ReportPortal Documentation for steps on how to backup and restore binary data

2. Follow Upgrading PostgreSQL for ReportPortal v24.2 and later for steps on how to backup and restore Postgres

Let’s go 🏃

Backup postgres

ReportPortal keeps most of the data in a Postgres instance including launches, filters, widgets, and dashboards. We can execute the pg_dump command inside the docker container to backup Postgres. This generates an SQL file with all the information needed to restore data after the upgrade.

DB_USER=rpuser
DB_NAME=reportportal
DB_PASSWORD=rppass
DB_CONTAINER=postgres

docker exec -e PGPASSWORD=$DB_PASSWORD $DB_CONTAINER pg_dump -U $DB_USER -d $DB_NAME > reportportal_docker_db_backup.sql

Explanation:

• We first export required parameters like DB_USER, DB_PASSWORD, DB_NAME, and DB_CONTAINER to ensure what can vary is captured upfront and we have an easy-to-read command. In this example, I’m using the default values that you can see from the docker-compose.yml file here. You may want to change these and maintain another docker-compose.yml file

• docker exec -e PGPASSWORD=$DB_PASSWORD $DB_CONTAINER: Executes a command inside the specified Docker container while setting the Postgres database password upfront as an environment variable (using -e)

• pg_dump -U $DB_USER -d $DB_NAME: This command initiates a PostgreSQL dump, specifying the username and database name.

• > reportportal_db_backup.sql: Redirects the output sql to a file named reportportal_docker_db_backup.sql.

⚡Tip: This command may take time if the database has lots of data. Please be patient as that completes. Do not exit out early as you may have incomplete backup. Once this is completed, it would be a good idea to make a copy of this file by executing something like this:

cp reportportal_docker_db_backup.sql YYYY_MM_DD_reportportal_docker_db_backup.sql

This will help avoid surprises if you somehow corrupt or delete the primary backup.

Backup binary data

Next, we take a backup of binary data. ReportPortal uses minIO as the object store to store additional data similar to how a file system would.

VOLUME_NAME=reportportal_storage

docker run --rm -v "$VOLUME_NAME":/data -v "$(pwd)":/backup busybox tar -zcvf /backup/reportportal_storage_backup.tar.gz /data

Explanation:

• docker run: This command runs a Docker container.

• --rm: This flag removes the container once it completes its execution. This is useful for temporary containers, such as when performing a one-off task like creating a backup.

• -v "$VOLUME_NAME":/storage_backup: This option creates a volume mount between the host machine and the container. It mounts the Docker volume named by the VOLUME_NAME variable to the path /storage_backup inside the container.

• -v "$(pwd)":/backup: This option mounts the current working directory ($(pwd)) on the host machine to the path /backup inside the container. This is where the backup file will be stored.

• busybox: This specifies the Docker image to be used for the container. In this case, it uses the BusyBox image, which is a lightweight and minimalistic Linux distribution often used for simple tasks.

• tar -zcvf /backup/reportportal_storage_backup.tar.gz /data: This part of the command runs the tar command inside the container to create a compressed tar archive (reportportal_storage_backup.tar.gz).

Remove all containers

We will now use docker-compose to bring down all existing containers. If any container is already running when you try to upgrade it, you must stop and manually remove it (using docker stop <container_name> && docker rm <container_name>)

This can be a tedious process to do one by one, as there could be dependencies between containers. Docker Compose simplifies this process for us

docker compose -p reportportal down

Remove Postgres volume

This step is optional, you should run this only if you want a clean slate in your DB. This would remove the existing Postgres volume.

docker volume rm reportportal_postgres

Make changes in your docker-compose.yml file

Now that everything has been torn down, you can change the docker-compose.yml file.

For example, below are a few use cases when this may be relevant

Upgrade versions

You may want to pull a new version of ReportPortal

curl -LO https://raw.githubusercontent.com/reportportal/reportportal/master/docker-compose.yml

Expose Postgres db

You may want to expose Postgres running in the container to the host machine to allow engineers to connect to it and make use of stored data for their purposes

## Uncomment to expose Database
ports:
   - "5432:5432"

## PostgreSQL as the main database for ReportPortal
postgres:
 image: bitnami/postgresql:16.4.0-debian-12-r7
 container_name: *db_host
 logging:
   <<: *logging
 shm_size: '512m'
 environment:
   POSTGRES_USER: *db_user
   POSTGRES_PASSWORD: *db_password
   POSTGRES_DB: *db_name
   POSTGRESQL_CHECKPOINT_COMPLETION_TARGET: 0.9
   WORK_MEM: 96M
   WAL_WRITER_DELAY: 20ms
   SYNCHRONOUS_COMMIT: off
   WAL_BUFFERS: 32MB
   MIN_WAL_SIZE: 2GB
   MAX_WAL_SIZE: 4GB
 volumes:
   - postgres:/bitnami/postgresql
 ## Uncomment to expose Database
 ports:
   - "5432:5432"
 healthcheck:
   test: [ "CMD-SHELL", "pg_isready -d $$POSTGRES_DB -U $$POSTGRES_USER" ]
   interval: 10s
   timeout: 120s
   retries: 10
 networks:
   - reportportal
 restart: always

Add OpenSearch dashboards

You may want to add the capability also to serve open search dashboards so that I can add custom search/visualization capabilities on top of what ReportPortal provides out of the box

opensearch:
 image: opensearchproject/opensearch:2.16.0
 container_name: *opensearch_host
 logging:
   <<: *logging
 environment:
   discovery.type: single-node
   plugins.security.disabled: "true"
   bootstrap.memory_lock: "true"
   OPENSEARCH_JAVA_OPTS: -Xms512m -Xmx512m
   DISABLE_INSTALL_DEMO_CONFIG: "true"
 ulimits:
   memlock:
     soft: -1
     hard: -1
 ## Uncomment the following lines to expose OpenSearch on ports 9200 and 9600
 ports:
   - "9200:9200"  # OpenSearch HTTP API
   - "9600:9600"  # OpenSearch Performance Analyzer
 volumes:
   - opensearch:/usr/share/opensearch/data
 healthcheck:
   test: [ "CMD", "curl","-s" ,"-f", "http://0.0.0.0:9200/_cat/health" ]
 networks:
   - reportportal
 restart: always


opensearch-dashboards:
 image: opensearchproject/opensearch-dashboards:2.16.0
 container_name: opensearch-dashboards
 ports:
   - "5601:5601"  # Dashboard UI
 environment:
   OPENSEARCH_HOSTS: http://opensearch:9200
   DISABLE_SECURITY_DASHBOARDS_PLUGIN: "true"  # Add this line to disable security
   OPENSEARCH_ALLOW_INSECURE: "true"          # Add this line to allow insecure connections
 depends_on:
   - opensearch
 networks:
   - reportportal
 restart: always

Start only postgres

After your changes are done, we’ll first only bring up the database

docker compose -p reportportal up -d postgres

Restore Postgres

Next, let’s restore the data. To restore the data we use psql

DB_USER=rpuser
DB_PASSWORD=rppass
DB_NAME=reportportal
DB_CONTAINER=postgres

docker exec -i -e PGPASSWORD=$DB_PASSWORD $DB_CONTAINER psql -U $DB_USER -d $DB_NAME < reportportal_docker_db_backup.sql > upgrade_db.log 2>&1

Explanation

1. docker exec This runs a command inside a running Docker container.

   1. -i: Keeps the input stream open so the SQL script can be passed to the psql command.

   2. -e PGPASSWORD=$DB_PASSWORD: Sets the PGPASSWORD environment variable inside the container. PostgreSQL uses this variable for authentication, so you don’t need to type the password interactively.

   3. $DB_CONTAINER: Refers to the name of the container (postgres).

2. psql is the PostgreSQL command-line tool. It connects to the database and executes SQL commands.

   1. -U $DB_USER: Specifies the database username (rpuser).

   2. -d $DB_NAME: Specifies the database name to connect to (reportportal).

   3. < reportportal_docker_db_backup.sql: Redirects the contents of the SQL backup file (reportportal_docker_db_backup.sql) as input to psql. This restores the database from the backup file.

3. Redirect Output with > upgrade_db.log 2>&1

   1. >: Redirects the standard output of the command (success messages, etc.) to a file called upgrade_db.log.

   2. 2>&1: Redirects the standard error (error messages) to the same file (upgrade_db.log) as the standard output. This captures all logs (success and errors) into a single file.

To verify the data is indeed restored, we should check the contents of the upgrade_db.log file

Also, you can connect to the postgres db server

Connect to the docker container

docker exec -it postgres bash

Open psql

psql -U rpuser -d reportportal

When prompted for the password, enter the postgres password from the docker-compose.yml file or the new one if you have reset it

Check if the below tables have values:

SELECT * FROM test_item limit 20;

SELECT * FROM test_item_results limit 20;

Start all other services

Next, we should bring up all the other containers

docker compose -p reportportal up -d

Restore binary data

Finally, let's restore binary data as well

VOLUME_NAME=reportportal_storage

docker run --rm -v $VOLUME_NAME:/data -v $(pwd):/backup busybox tar -xzvf /backup/reportportal_storage_backup.tar.gz -C /

We follow a similar process as the above to restore the gzip tar backup that we had taken back into minIO

Explanation

1. docker run

   • This command runs a temporary container from a specified image (busybox).

2. --rm

   • Automatically removes the container after it finishes running. This prevents leftover containers from cluttering the system.

3. -v $VOLUME_NAME:/data

   • Mounts a Docker volume named $VOLUME_NAME to the /data directory inside the container.

   • This allows the container to interact with the Docker volume's data.

4. -v $(pwd):/backup

   • Mounts the current working directory (represented by $(pwd)) to the /backup directory inside the container.

   • This makes files in the current directory, like the backup file, accessible to the container.

5. busybox

   • Specifies the Docker image to use for the container.

   • busybox is a lightweight image commonly used for basic utilities like file operations.

6. tar -xzvf /backup/reportportal_storage_backup.tar.gz

   • tar is a command-line tool used to extract files from archives.

     • -x: Extracts the files.

     • -z: Handles .gz (gzip-compressed) files.

     • -v: Enables verbose mode, showing the files being extracted.

     • -f: Specifies the input file, in this case, /backup/reportportal_storage_backup.tar.gz.

7. -C /

   • Changes the target directory for extraction to / (the root directory inside the container).

   • Files from the archive are extracted to their corresponding locations in /.

And you are done! 🎉

Congratulation! You have successfully managed to take a backup, upgrade the instance, and restore it all back now.

To verify login to ReportPortal http://localhost:8080/ui/ and check your launches, filters, and dashboards are all available

Connect to Postgres from the host machine

To verify if we can connect to Postgres running inside the docker container, you will need postgres on your local machine, on mac you can use homebrew to install

brew install postgresql@16

# Postgresql
export PATH="/opt/homebrew/opt/postgresql@16/bin:$PATH"
export LDFLAGS="-L/opt/homebrew/opt/postgresql@16/lib"
export CPPFLAGS="-I/opt/homebrew/opt/postgresql@16/include"

Then start the postgres service

brew services start postgresql@16

You can connect to Postgres from any client (like DBeaver) by using the below details

jdbc:postgresql://192.168.29.131:5432/reportportal
User name: rpuser
Password: rppass

Or, from the host machine, you can also connect to the PostgreSQL instance using psql to ensure the port is accessible:

psql -h localhost -p 5432 -U rpuser -d reportportal

OpenSearch dashboards

To test access to OpenSearch, you can hit http://<host_ip>:9200

This should print details about the OpenSearch cluster

{
  "name": "0fdbaba7ac12",
  "cluster_name": "docker-cluster",
  "cluster_uuid": "R0DM_scOSRuB0kba7Q6rGw",
  "version": {
    "distribution": "opensearch",
    "number": "2.16.0",
    "build_type": "tar",
    "build_hash": "f84a26e76807ea67a69822c37b1a1d89e7177d9b",
    "build_date": "2024-08-06T20:32:34.547531562Z",
    "build_snapshot": false,
    "lucene_version": "9.11.1",
    "minimum_wire_compatibility_version": "7.10.0",
    "minimum_index_compatibility_version": "7.0.0"
  },
  "tagline": "The OpenSearch Project: https://opensearch.org/"
}

Finally, you can open OpenSearch Dashboards via http://<host_ip>:5601

Summary

Let’s conclude on the workflow we learned today

1. Backup Postgres: Use the pg_dump command inside the Docker container to create an SQL file of your database.

2. Backup Binary Data: Create a compressed tar archive of your Minio data using the Docker run command.

3. Remove Existing Containers: Use docker-compose down to stop and remove all running containers. (Optional: Remove Postgres volume for a clean database.)

4. Modify docker-compose.yml: Update versions, expose ports, or add new components like OpenSearch.

5. Start Postgres: Use docker-compose up to start the Postgres container.

6. Restore Postgres Data: Use the psql command to restore data from the backup SQL file.

7. Start Remaining Services: Bring up all other containers with docker-compose up.

8. Restore Binary Data: Extract the tar archive into the Minio volume using the Docker run command.

9. Verify: Check ReportPortal, Postgres, and OpenSearch to ensure data and functionality are restored.

Thanks for the time you spent reading this 🙌. If you found this post helpful, please subscribe to the newsletter and follow my YouTube channel (@automationhacks) for more insights into software testing and automation. Until next time 👋, Happy Testing 🕵🏻 and Learning! 🌱| Substack | YouTube | Blog | LinkedIn | X | BlueSky