AI Builder Deep Dive: Overview of Possibilities and Best Practices

Have you ever had to manually input data from a document into a database or Excel sheet? Whether you’ve tried using code-based crawlers or complex AI monitoring to extract this data automatically, the barrier to entry for AI operations on documents was fairly large and previously required a developer to create the tools. For Power Platform users, that barrier to entry no longer exists! There’s a magical tool I’ve been using extensively that’s available to the whole Power Platform that may be more individually impactful to data input automation than anything before it: AI Builder! Fantastic overviews of the product already exist, so today we’ll dive into a basic overview of the platform, specific use cases, and best practices to help you get started.

AI Builder Overview

AI Builder is a tool available to all Power Platform products that aids in data extraction, prediction, and classification. Power Platform’s goal is to provide products that replace what used to require a full-stack developer. In school, I learned about web crawlers and how you could extract text and images from websites using prebuilt libraries. These libraries required some background in object-oriented programming which generally isn’t the case for the people actually working with the data. AI Builder… builds off of the concept of web crawlers and makes the output available to any tool that could assist with task automation.

Using prebuilt and custom models, AI Builder provides a suite of options that tackle document and text extraction, structured data analytics, and image detection. My favorite use case to distinguish prebuilt vs custom models is the invoice extraction options provided to you out of the box with AI Builder:

Prebuilt Model

With the prebuilt invoice processing model, users can retrieve any invoice on the platform of their choosing (my favorite is the SharePoint Document library or from a received email attachment) and extract information from it using the “Extract information from invoices” action in Power Automate .

After running that action, a variety of predetermined fields will become available to you, along with a confidence score for each one. The confidence score is baked into the prebuilt model as a way of determining if the model can confidently say it found the field listed. If your invoice is clear and easy to read, these scores should be high consistently. From there, you can use any of these fields to write automailers, create table items in Dataverse, or reach out to any of the hundreds of premium connectors to third-party sites available in the Power Platform.

The prebuilt model is great but there are limitations. The most notable is you are beholden to the predetermined set of fields it deems as regularly available in an “invoice”. If you are brand new to Power Automate, it is a great tool to plug-and-play with because it requires no manual input. You provide a file, and the model finds what it can. And there are prebuilt models available to you for text classification, predictions, and image descriptions as well as documents. But what if your invoice is more complex and uses industry-specific fields that aren’t in the list of fields available for this prebuilt model?

Custom Model

Custom models are built by the user to specifically target the documents, text, and images necessary to analyze for their business needs. Let’s a drilling business receives an invoice every single day with information related to say one of their wells. This invoice includes a daily log of the activities, associated costs, and layers of technical jargon only someone in the industry would understand. Using a custom model, a Power Platform developer can determine what fields exist in these documents and manually train the model to find those fields. Using structured documents, the user can specifically tell the model where to look using past invoices and the model will attempt to extract data based on what the user tells it. Training the model is as easy as dragging boxes over where text fields and tables are in the document. Here’s a breakdown of the process:

1. Give the model what fields it’s looking for.

2. Point the model towards those fields using documents that you upload (or that already exist in Power Platform).

3. Train the model. This should take a few minutes generally but may take longer if the breadth of fields is large. After it finishes training, you can return to the model dashboard and see your accuracy score, which you will see is individually tied to the fields it’s looking for.

When you run this model using Power Automate, all of the fields you listed as available in the document will be available to pull for your automation needs, as well as confidence scores. If your business currently has somebody manually inputting invoices into a database, this tool can save hundreds of hours that could be better spent on innovation. Similar custom models are available for text and image categorization.

Prebuilt vs Custom

Both options have their pros and cons. Prebuilt models are fantastic for time optimization; you can spend less time training the technology and it will be available to you right away. It’s great for bulk text extraction, language detection, data predictions, and common classifications. If your business has the time and resources to train custom models, they may better serve your needs for complex data extraction, diverse text categorization, object detection within images, and more. Either way, using these tools will save your triage team time.

Best Practices

Like most products available on the Power Platform, we suggest you do some testing before using these tools in production. AI Builder is a bit unique because it will not affect output until you use the data available. Testing can happen in production because you are simply extracting and analyzing data, you are not altering it in any way. However, testing in a development environment is a great way to provide a proof-of-concept before you’re ready to start using it with live data. This obviously only applies to custom models, as the prebuilt models will be the same in any environment. If you’d like to move a custom model from the development environment to your live environment, you will:

  1. Go to make.powerapps.com and go to your development environment
  2. Click on the “Solutions” tab
  3. Click “New Solution” in the top ribbon
  4. Create a new solution and add your model to it by clicking “Add existing” in the ribbon
  5. Click “Publish all customizations” in the ribbon

This will automatically download a ZIP file. If you’re moving from development to production, you’ll want to export a Managed Solution. There is one caveat: once you export the model in a managed solution, You will not be able to train the model any further in your target environment. The best practice is to have a good confidence score on your model before exporting. After you import it into your target environment, you will be able to use that model and start automating!

As for licensing and cost, a few Power Apps and Power Automate licenses have service credits available for AI Builder. You will need to contact your account lead to work with Microsoft and see what license makes the most sense for your business needs

In Conclusion

AI Builder is an expansive tool that can save your business hundreds of hours yearly. It can help your triage team with data input. It can help HR see if feedback coming back on a specific product is positive or negative. It can predict how busy December will be for your company based on what happened last year. It can even act as a tool to recognize hot dogs in your images (shout out to Silicon Valley if you’ve watched it). The possibilities are endless, so whether you have a trial available or credits in your environment already I suggest you go play with it and see how you can improve your business outcomes!

Below are some resources to help get you started:

.

Leave a Comment