Stick to your experimentation process
My team have been working on a tool called cloud agent 1 for automatic agentic workflows, and we made some mistakes regarding communication to the rest of the organization. I reflect what had happened, what the actual problem was and how we could have avoided it.
What happened
For a couple of weeks, my team is experimenting with an agent running on AWS. Soon after we have started, we informed our manager about it. After a first working prototype has been developed, we wrote a blog article and shared it in an AI related Slack channel. Next, we held a presentation on a Tech Day about the “State of AI in our Team” where we also talked about it. We even informed our CTO, CISO and our platform team about it. And still people were not happy how we communicated it.
The actual complaint that received us was simply: “You should have added it to the Tech Radar 2.” At first, this sounded strange to me: We used different formats to share knowledge about the cloud agent, but how should have a simple blip on our Tech Radar made the whole situation better?
What we actually missed
I think the complained itself didn’t grasp the real problem. It’s not the knowledge sharing that was missing, it was us not violating the experimentation process. By sharing information about the cloud agent, we created desire for other teams. They also have the demand for an autonomous agent, fulfilling their requests.
But none of the other teams got the approval from our CTO. He wanted to avoid that multiple teams do the same learnings (and especially the mistakes) twice. He wanted to make sure that we didn’t build an uncontrollable OpenClaw like tool, running out of control, acting maliciously and costing us a lot of money.
Other teams weren’t aware of the concerns of our CTO. For them, it looked like as they have been discriminated, because
my team got approval, and they didn’t. This was the real issue. We should have made it more clear that this
experiment needs to reach some certain level of maturity, until others can pick it and start using it. By putting it on
the Tech Radar in the trial blip, the level of maturity has been set for now.
How to avoid in future
I learned now to be careful when you publish promising and exciting results, because it awakens desire. We should have stuck to the process for experimentation and should have communicated it more defensively in the beginning.
The way forward is to make the cloud agent compliant in terms of documentation and also restricting its capabilities from a general purpose agent to well-defined use cases. We started with the use case of fixing automatically failed vulnerability pipelines.
I also think that we have the obligation to keep others in the loop, because my team was given the trust and approval to work on AI experiments. For the next experiment, I’ll make sure to comply with the experimentation process and to share knowledge with the rest of the organization regularly.