Artificial Solutions International (fka ASAIF) (ASAIF)

[StockpickerCAPC]

Testing your Teneo Conversational AI Solution with Botium.

source
https://www.artificial-solutions.com/blog/testing-your-teneo-conversational-ai-solution-with-botium

August 24, 2021

We are always looking to strengthen the capabilities of our Teneo platform
https://www.artificial-solutions.com/teneo
by seeking partnerships that help our customers enhance their solutions and boost productivity. Teneo is an open, customizable platform that allows customers to connect with their preferred tools (back-end and front-end systems, channels, RPA, etc.) throughout the different stages of development using Teneo Connectors.
https://developers.artificial-solutions.com/resources/backend-connectors/power-automate

In this article, we are focusing on the importance of end-to-end testing and why it is vital to ensure that chatbots function correctly and meet the expectations that have been bestowed on them. While auto-testing in Teneo allows you to perform quality assurance tasks during the development and maintenance of a Teneo solution, you may wish to perform end-to-end tests before you release your bot to ensure the bot behaves as intended, the answers provided are as expected and the integrations work as anticipated. For this reason, we’ve partnered with Botium.ai
https://www.botium.ai/
to create a connector
https://developers.artificial-solutions.com/studio/test-your-bot/test-your-bot-with-botium
that allows developers to use their product for end-to-end testing solution. Botium is a quality assurance framework that was specifically developed for regression testing of chatbots, including Teneo chatbots.
So, if you want to know more about how to start testing your Teneo chatbots with Botium, this post is for you.

In case you are new to using Teneo, the first thing you need to do is register to get a free sandbox environment at
https://developers.artificial-solutions.com/signup/?utm_source=botium&utm_medium=article0821
o get your first solution up and running you can follow the following tutorial:
https://developers.artificial-solutions.com/getting-started/your-first-solution
After you have your solution running you move on to testing with Botium.

Start testing with Botium
The first step is indeed to log into your Teneo Studio and make sure that you have published the chatbot you’d like to work with. Once you have accomplished this step, you will get a URL that opens the chatbot you have built in a sandbox environment. After that, your Botium Box account is waiting for you to set up the first smoke test.


?? Regression testing
is done to make sure that the new deployments or changes in the conversation flow did not have any negative side effects on the existing functionalities. It is nothing but a full, or partial selection of already executed test cases, which are re-executed to ensure existing functionalities work fine.

In this test type, Botium is testing directly against Teneo, directly against the NLP engine in use, or bot-building platform. This is the advisable starting point for testing conversational AI.

?? Why?
At API level, you can execute enormous test sets in just a few seconds, so the most burning problems will occur at this point. One important difference between traditional Software Testing and Chatbot Testing is the size of the test sets. Talking to a chatbot usually has no barriers (button-based bots are exceptions), as the user can say anything to a bot. This peculiarity might result in infinitely large test sets.

You start by selecting your chatbot channel in our Quickstart Wizard. There’s a card including the logo and description for every technology. Botium supports more than 45 technologies and one of them is Teneo, of course.
So, go for it

Once you have chosen the Teneo channel, you will need to copy the URL from Teneo Studio to connect to your chatbot. It is rather easy to check if the credentials and the connection have been set up, as you can simply say hello to your chatbot, and receive an answer in case everything is working fine. The next step in the Botium Quickstart Wizard is to always compose your tests. For a simple smoke test, you can use the visual test case editor.


?? Be familiar with the capabilities of your chatbot.
Before composing your test, you must be aware of the limitations of your bot, the intents it should recognize and the structure of the conversation flow behind it.

In this case, we know that the demo Barista bot we are currently using is a funny folk and can tell us some jokes. So, we can just simply ask him to entertain us. You can send every request to the bot in clear text, using the visual text case editor. No coding or scripting is required. The same is valid for the bot answer. You can assert for every possible reply, no matter if it’s just text or more fancy stuff, like cards, carousels, buttons, voice etc. We save the test project and execute the tests by hitting the start session button. You have about enough time to take a sip in your coffee until the results arrive on the test cases tab. If the bot replies as expected, then everything is green, the test passes and we get back the joke expected.


To take things from here one step further, we can easily integrate the tests in every continuous integration and delivery pipeline. For every single project created in Botium Box, we are also generating a webhook, curl and cli command that can be triggered from your pipeline.

CI/CD pipeline integration


?? What’s the benefit?
Whenever you do a commit, or any changes in your conversation model or your UI, you can trigger your test project. To showcase this solution, we are using Jenkins, but it is working with every pipeline orchestration tool.

In Jenkins, we create a new item. Click on the freestyle project and define when to execute the tests. Our suggestion is to time these tests when the git repository gets changed, but you can also just add (with cron expressions) a specific time slot. The last thing we have to do in Jenkins is to add a built step. Therefore we could just say execute a shell command and so we go back to our Botium Box and copy the generated curl into Jenkins. Apply and Save! This means that from now on, these tests will be executed (in our case at 8:30), with the results being delivered both back to Jenkins and Botium. This small smoke test is of course far away from an entire regression testing, where we need to cover all possible paths through our conversation model, but a great start!

Let’s take things further with E2E testing
Let’s emphasize our efforts to make test automation fast and easy. For E2E testing we go through our Quickstart Wizard again. As usual, we select our chatbot channel, compose or choose our test sets and tell Botium where to run the tests E2E.


?? The main purpose of E2E Testing is to test from the end user’s perspective by simulating the real user scenario and validating the system under test and its components for integration and data integrity.

You can run these tests in Botium against a browser farm to test all your preferred browser types and versions in parallel. Or, against a device cloud provider, to test on all your clients preferred smartphones and tablets. Botium Box integrates with major device cloud providers. Or, for a simple smoke test it might be already sufficient to use the headless Chrome included in Botium Box. To shake things up, this time we show you how you can compose tests by recording your own voice. After finishing the Quickstart Wizard, Botium Box will open a live chat session. After you have a conversation with your bot, you can just save it as a new test case and execute them anytime you want. Botium also takes screenshots of the conversation, enabling you to have a look at the chat happening in the real environment.


To run the tests E2E, Botium Box will use Selenium and Appium in the background. In spite of that, you don’t need to write a single line of code. Botium will generate all the tests based on your chat recording.

We recently co-hosted a webinar in which you can see a demo of live E2E testing of a Teneo solution using Botium.
To watch it on-demand just click here.
https://www.brighttalk.com/webcast/17594/484772?utm_source=ArtificialSolutions&utm_medium=brighttalk&utm_campaign=484772