Power CAT Tools – Initial thoughts on using DocuPilot to “generate high-quality technical documentation effortlessly”

While preparing this quick blog post I realised I actually never published anything here about PowerDocu, my free tool that can help you to generate technical documentation of your Canvas Apps, Power Automate Flows, and Power Platform solutions generally – I really need to write more about it here soonest!

But coming to the main topic of this post – Microsoft’s Power CAT team just released a new suite of tools which include (among others):

• AI Solution Documentation – Generate high-quality technical documentation effortlessly.
• Code Review – Ensure high-quality code with insights and recommendations.
• Security Risk Assessment – Identify and mitigate security role risks proactively

Naturally, the first item is of upmost interest to me, as PowerDocu does something similar (minus the AI part) for the past 3+ years. I’ll try to cover more details between the two different tools in the upcoming days, for now I’ll simply share my first impression of DocuPilot, as the documentation tool has been called

Installation

Installation of the Power CAT Tools is relatively easy, as it can be done directly from AppSource. You simply click on “Get now” on https://appsource.microsoft.com/sr-latn-rs/product/dynamics-365/microsoftpowercatarch.powercattools?tab=overview, select your destination environment, and get the solution installed into it.

Launching Power CAT Tools

To launch the Power CAT Tools, simply start the Model-Driven App called Power CAT Tools (well, surprise) from the environment that you installed the solution in. Here’s the home screen which gives you an overview of the various tools:

When you select DocuPilot (AI Documentation in the navigation) for the first time, you are asked to allow access to the Office 365 Outlook and Dataverse connectors:

As the solution uses Dataverse to store information, the request for this connector is clear. I wasn’t sure where the Office 365 Outlook connector gets used, something I need to review in more detail soonest.

I had no solutions documented so far (of course), so I saw this blank screen

Once you click on New, you have two options: either select a solution in your current environment (which means that if you plan to use DocuPilot across multiple environments, you need to install it in each environment), or upload a solution to have it documented:

I tried to document the solutions in my environment, but all I saw was “Reading solution” that never went away and no solution documentation process was started.

So, I decided to upload a solution, and selected the Core Components solution from the CoE Starter Kit:

Within a couple of seconds it started the documentation process – I assume it read the solution (which may be quite fast or take a short while, depending on how complex the solution is), sent its details to an AI endpoint to understand and explain it, and then took the response and turned it into the final documentation:

The final documentation that was generated was however a bit underwhelming. At the top you have a summary which I assume is automatically generated by AI. Below that is a very plain extract listing some of the solution components (e.g. apps, Flows, tables) but not all (data sources, environment variables, others).

I saved two generated documentations and tried to open again from the overview screen – strangely, all buttons were disabled and I could not access the documents again:

Final Thoughts

At a first glance, DocuPilot is relatively disappointing. While first versions are usually quite basic (the very first version of PowerDocu could just about document Power Automate Flows, for example), I was hoping for a bit more here. The generated solution description is the most interesting aspect of it so far, as this is something that is missing in PowerDocu (which doesn’t leverage any AI capabilities).

The documentation generated for the rest of the solution is in my opinion not too valuable as of right now. For example, when you look at what you get as output for a Dataverse table, you only have the table name, description, and a list of internal names of the columns – no details on display name, column type, etc. I am not sure who the target audience for this is at the moment – the high-level overview doesn’t contain as much as needed, and the details that a developer would look for simply aren’t there.

Overall, I am not too tempted to use DocuPilot in its current form. Having said that, I am however very happy to see that Microsoft is working on something that will hopefully result in a useful version in the future. Having a documentation tool that is accessible directly within the platform (DocuPilot) instead of outside as a standalone application (PowerDocu) does sound very tempting in various ways.
So, looking forward to seeing and testing future enhancements!