Your most important tools in the continuing development of your bot are your chat logs. These logs provide conversation-by-conversation documentation of your users’ interactions with your bot.

First, they allow you to identify ways in which your users are asking questions about the topics you’ve covered that you have missed. It’s virtually impossible for the programmer to think of and program all the possible formulations of a given inquiry into the initial AIML coding. Chat logs offer you a documentation of query formulations you have missed. These alternate formations of questions allow you to refine your use of wildcards and <srai> tags to most efficiently and comprehensively address your topics.

In addition, chat logs provide documentation of areas where library jargon and language may be getting in the way of sharing information with your users. An example is the current flood of interest in e-media. While librarians may be thinking in terms of “downloadables” or “digital media” because they are trying to be inclusive of various types of media (e-books, e-audiobooks, and downloadable video and music files), many users are simply looking for “e-books.” Looking at the language your users are most familiar with by documenting their inquiries allows you to offer alternate paths to the information they need. At the same time, this information allows you to use your template responses to inform users about the variety of resources you have available. For example, information about the variety of alternate digital media available can be included in your responses to inquiries about e-books.

Finally, reviewing the chat logs can provide you with valuable insight into new thematic areas that need development. If you see you are getting questions about an area that has not heretofore been addressed by your website or suite of services, you may consider expanding your offerings in this area to meet these needs.

Thus the importance of reviewing the chat logs cannot be overstated, both for refining and correcting omissions in your bot’s existing knowledge base and for expanding that knowledge base into areas of perceived need.

The most simple and comprehensive approach to chat log review is to look at each logged conversation to identify the success of each inquiry-and-response pair. This allows you to both monitor the logs for needed changes (as noted above) and to document the percentage of correct responses. Though time-consuming, this is probably the best approach for newly activated bots. This detailed attention to the users’ interaction with the new bot will greatly enhance your ability to customize and improve your bot’s functioning.

In Pandorabots, there is a special interface provided to monitor conversation logs. Using this interface you can view each conversation in its entirety. Figure 7.1 shows the Pandorabots Conversation Logs interface.

If you identify specific responses you’d like to change, simply click the Train button to the left of the response (figure 7.2). The Training interface (figure 7.3) appears and you can adjust the response as needed.

Keep in mind that these changes are specific to the matched formulation of a question. It may be more effective to make changes to the core AIML files to incorporate major adjustments rather than fixing specific categories in a piecemeal fashion.

As your bot matures and your correct response rates improve, you may wish to turn to a more selective approach to reviewing your logs. AIML Targeting, offered by Pandorabots, is a process that allows the botmaster to scan the chat logs for specific patterns. This allows the botmaster to look for particular responses and view all the inquiries that elicit those responses. For example, viewing all the inputs that call the single wildcard * category will show you those questions to which you have programmed no response. Scanning these inputs, you may identify many that are just gibberish or typing errors, but you may also find legitimate questions to which you want to compose responses.

To use the Targeting features, select conversations you wish to evaluate, select Find Targets from the drop-down menu below the list of conversations (see figure 7.4) and click OK.

Click Find Targets at the bottom of the following screen. (Once you’re conversant with using the targeting features you may wish to filter the results or merge them with previous target sets, but leave these options all unchecked for now.) Figure 7.5 shows the Find Targets screen.

The next screen (figure 7.6) will display all the response patterns activated in the conversations selected with the most frequently called responses listed first.

You will likely find that the single wildcard character appears as one of the most frequently called patterns. Click the count number to the right of the pattern you wish to investigate to see all the relevant inputs, their associated topics, and the response to the previous input (displayed in the That section of the table; see figure 7.7). From this point you can use the training interface to adjust the responses to the selected input.

Finally, since the response patterns are displayed in the Targeting interface in the order of the frequency with which they’ve been called, this interface can show you which topics and themes are receiving the most inquiries.

As mentioned before, the value of the chat logs in identifying topics or specific questions that elicit incorrect, vague, or useless responses cannot be overstated. Though it’s true that this makes the chatbot a perennial work in progress, it also allows the bot and its knowledge base to continue to grow and reflect more fully the changing suite of resources and services provided by the library.