How to arrange to jot down your first Mycroft AI talent utilizing Python

With the latest worldwide pandemic and stay-at-home orders, I’ve been in search of issues to do to switch a few of my regular actions. I began to replace my house electronics setup and, as a part of that, to delve into house automation. Some of my pals use Amazon’s Alexa to show lights on and off of their home, and that’s interesting on some degree. However, I’m a privacy-conscious particular person, and I used to be by no means actually comfy with gadgets from Google or Amazon listening to my household on a regular basis (I am going to ignore cellphones for the sake of this dialog). I’ve recognized in regards to the open supply voice assistant Mycroft for about 4 years, however because of early struggles with the venture, I would by no means investigated it too intently. The venture has come a really great distance since I first stumbled throughout it, and it checks quite a lot of packing containers for me:

• Self-hosted
• Easy onboarding (through Python)
• Open supply
• Privacy-conscious
• Interactive chat channel

In the first article on this sequence, I launched Mycroft, and within the second article, I touched upon the idea of expertise in synthetic intelligence. In its most elementary kind, a talent is a block of code that’s executed to realize the consequence desired for an intent. Intents try to find out what you need, and a talent is the best way Mycroft responds. If you’ll be able to consider an consequence, there’s in all probability a strategy to create a talent that makes it occur.

At their coronary heart, Mycroft expertise are simply Python applications. Generically, they’ve three or 4 sections:

1. The import part is the place you load any Python modules required to perform the duty.
2. An elective operate part comprises snippets of code which are outlined exterior of the primary class part.
3. The class part is the place all of the magic occurs. A category ought to at all times take the MycroftAbility as an argument.
4. The create_skill() part is what Mycroft makes use of to load your talent.

When I write a talent, I usually begin by writing a regular Python file to make sure my code does what I feel it does. I do that primarily as a result of the workflow that I’m used to, together with debugging instruments, exists exterior of the Mycroft ecosystem. Therefore, if I have to step by way of my code, I discover it far more acquainted to make use of my IDE (PyCharm) and its built-in instruments, however this can be a private desire.

All the code for this venture will be present in my GitLab repo.

About intent parsers

The talent on this venture makes use of each the Padatious and Adapt intent parsers, which I described in my previous article. Why? First of all, this tutorial is supposed to supply a concrete instance of a number of the options you may wish to think about using in your personal talent. Second, Padatious intents are very simple however don’t help regular expressions, whereas Adapt places regex to good use. Also, Padatious intents aren’t context-aware, which implies that, whilst you might immediate the person for a response after which parse it following some decision-tree matrix, you could be higher off utilizing the Adapt intent parser with Mycroft’s built-in context handler. Note that, by default, Mycroft assumes you might be utilizing the Padatious intent handler. Finally, it is good to notice that Adapt is a key phrase intent parser. This could make complicated parsing cumbersome if you’re not a regex ninja. (I’m not.)

Implement the three T’s

Before you begin writing a talent, contemplate the three T’s: Think issues by way of! Similar to whenever you’re writing a top level view for an essay, whenever you’re beginning to develop a talent, write down what you need your talent to do.

This tutorial will step by way of writing a Mycroft talent so as to add gadgets to the OurGroceries app (which I’m not affiliated with). In fact, this talent was my spouse’s thought. She wished an software she might use on her cellphone to handle her procuring lists. We tried nearly a dozen apps to attempt to meet our particular person wants—I wanted an API or a strategy to simply work together with the backend, and he or she had a large listing of standards, some of the necessary was that it’s straightforward to make use of from her cellphone. After she made her listing of Must-haves, Nice-to-haves, and Wish-list gadgets, we settled on OurGroceries. It doesn’t have an API, nevertheless it does have a strategy to work together with it by way of JSON. There is even a handy library known as py-our-groceries in PyPI (which I’ve contributed some small quantity to).

Once I had an goal and a goal platform, I began to stipulate what the talent wanted to do:

1. Login/authenticate
2. Get an inventory of the present grocery lists
3. Add merchandise to a particular grocery listing
4. Add merchandise to a class underneath a particular listing
5. Add a class (since OurGroceries permits gadgets to be positioned in classes)

With this in thoughts, I began to sketch out the required Python. Here is what I got here up with.

Create the Python sketch

By studying the examples for the py-our-groceries library, I discovered I wanted to import simply two issues: asyncio and ourgroceries.

Simple sufficient. Next, I knew that I wanted to authenticate with username and password, and I knew what duties this system wanted to do. So my sketch ended up wanting like this:

import asyncio
from ourgroceries import OurGroceries
import datetime
import json
import os
USERNAME = ""
PASSWORD = ""
OG = OurGroceries(USERNAME, PASSWORD)
def fetch_list_and_categories():
    move
def return_category_id():
    move
def add_to_my_list():
    move
def add_category():
    move

I will not go into the complete particulars of what makes this sketch tick, as that’s exterior the scope of this sequence. However, in order for you, you’ll be able to view the working outline in its entirety.

Before you’ll be able to start programming, it is advisable to have your username, password, and an inventory ID. The username and password are apparent. The listing ID will be retrieved from the URL after clicking on the hyperlink, or extra programmatically, you should use the Developer Tools in your browser of selection and examine the objects. Here is what the Developer Tools seems like in Firefox:

Once you could have an inventory ID, log into OurGroceries and get a cookie. To do that, create an OurGroceries object after which move it into asyncio. While you might be at it, you may as nicely outline your listing ID, as nicely:

OG = OurGroceries(USERNAME, PASSWORD)
asyncio.run(OG.login())
MY_LIST_ID = "a1kD7kvcMPnzr9del8XMFc"

For the needs of this venture, it is advisable to outline two object varieties to assist manage your code: groceries and classes. The fetch_list_and_categories methodology is fairly simple:

def fetch_list_and_categories(object_type=None):
    if object_type == "groceries":
        list_to_return = asyncio.run(OG.get_list_items(list_id=MY_LIST_ID))
    elif object_type == "categories":
        list_to_return = asyncio.run(OG.get_category_items())
    else:
        list_to_return = None
    return (list_to_return)

OurGroceries lets you add a couple of class or merchandise with the identical identify. For instance, if you have already got “Meat” in your listing and also you add it once more, you will note a class known as “Meat (2)” (this quantity increments everytime you create a class with the identical identify). For us, this was undesirable conduct. We additionally wished to keep away from duplication as a lot as potential, so I made a rudimentary try at detecting plurals; for instance, my code checks for each “Meat” and “Meats.” I’m positive there’s a extra clever means of performing these checks, however this instance highlights a number of the issues you might wish to take into consideration as you progress. For brevity, I’ll omit these checks, so the return_category_id methodology seems one thing like this:

def return_category_id(category_to_search_for, all_categories):
    category_to_search_for_lower = category_to_search_for.decrease()
    category_id = None
    if len(all_categories['listing']['gadgets']) is not zero:
        for category_heading in all_categories['listing']['gadgets']:
            # Split the heading as a result of if there's already a reproduction it
            # presents as " (2)"
            category_heading_lowered = category_heading['worth'].decrease().break up()[zero]
            if category_to_search_for_lower == category_heading_lowered:
                category_id = category_heading['id']
                break
    return(category_id)

To add an merchandise to the listing, you wish to:

1. Check that the merchandise doesn’t exist already
2. Obtain the class ID
3. Add the merchandise to the listing underneath a particular class (if specified)

The add_to_my_list methodology finally ends up one thing like this:

def add_to_my_list(full_list, item_name, all_categories, class="uncategorized"):
    # verify to verify the thing would not exist
    # The groceries dwell in my_full_list['list']['items']
    # Start with the idea that the meals doesn't exist
    food_exists = False
    toggle_crossed_off = False
    category_lowered = class.decrease()
    for food_item in full_list['listing']['gadgets']:
        if item_name in food_item['worth']:
            print("Already exists")
            food_exists = True
    if not food_exists:
        category_id = return_category_id(category_lowered, all_categories)
        asyncio.run(OG.add_item_to_list(MY_LIST_ID, item_name, category_id))
        print("Added item")

Finally, add_category runs the asyncio command to create a class if it doesn’t exist already:

def add_category(category_name, all_categories):
    category_id = return_category_id(category_name, all_categories)
    if category_id is None:
        asyncio.run(OG.create_category(category_name))
        refresh_lists()
        print("Added Category")
    else:
        print("Category already exists")

You ought to now be capable of check your sketch to verify every thing in every operate works. Once you might be happy with the sketch, you’ll be able to transfer on to eager about the way to implement it in a Mycroft talent.

Plan the Mycroft talent

You can apply the identical rules you used to sketch out your Python to growing a Mycroft talent. The official documentation recommends utilizing an interactive helper program known as the Mycroft Skills Kit to arrange a talent. mycroft-msk create asks you to:

• Name your talent
• Enter some phrases generally used to set off your talent
• Identify what dialog Mycroft ought to reply with
• Create a talent description
• Pick an icon from fontawesome.com/cheatsheet
• Pick a shade from mycroft.ai/colours or color-hex.com
• Define a class (or classes) the place the talent belongs
• Specify the code’s license
• State whether or not the talent could have dependencies
• Indicate whether or not you wish to create a GitHub repo

Here is an illustration of how mycroft-msk create works:

After you reply these questions, Mycroft creates the next construction underneath mycroft-core/expertise/<talent identify>:

├── __init__.py
├── locale
│   └── en-us
│       ├── ourgroceries.dialog
│       └── ourgroceries.intent
├── __pycache__
│   └── __init__.cpython-35.pyc
├── README.md
├── settings.json
└── settingsmeta.yaml

You can ignore most of those recordsdata for now. I desire to verify my code is working earlier than making an attempt to get into Mycroft-specific troubleshooting. This means, if issues go incorrect later, it’s associated to how your Mycroft talent is constructed and never the code itself. As with the Python sketch, check out the define that Mycroft created in __init__.py.

All Mycroft expertise ought to have an __init__.py. By conference, all code ought to go on this file, though if you’re a talented Python developer and understand how this file works, you may select to interrupt your code out.

Inside the file Mycroft created, you’ll be able to see:

from mycroft import MycroftAbility, intent_file_handler
class OurGroceries(MycroftAbility):
    def __init__(self):
        MycroftAbility.__init__(self)
    @intent_file_handler('ourgroceries.intent')
    def handle_test(self, message):
        self.speak_dialog('ourgroceries')
def create_skill():
    return OurGroceries()

In principle, this code will execute based mostly on the set off(s) you create through the msk create course of. Mycroft first tries to discover a file with the .dialog file extension that matches the argument handed to selfspeak_dialog(). In the instance above, Mycroft will search for a file known as ourgroceries.dialog after which say one of many phrases it finds there. Failing that, it should say the identify of the file. I am going to get extra into this in a follow-up article about responses. If you wish to do that course of, be at liberty to discover the assorted enter and output phrases you’ll be able to provide you with throughout talent creation.

While the script is a good place to begin, I desire to suppose by way of the __init__.py by myself. As talked about earlier, this talent will use each the Adapt and Padatious intent handlers, and I additionally wish to exhibit conversational context handling (which I am going to get deeper into within the subsequent article). So begin by importing them:

from mycroft import intent_file_handler, MycroftAbility, intent_handler
from mycroft.expertise.context import adds_context, removes_context

In case you might be questioning, the order you specify your import statements doesn’t matter in Python. After the imports are accomplished, take a look at the category construction. If you wish to study extra about lessons and their makes use of, Real Python has a fantastic primer on the topic.

As above, begin by mocking up your code with its supposed performance. This part makes use of the identical targets because the Python sketch, so go forward and plug a few of that in, this time including some feedback to assist information you:

class OurGroceriesSkill(MycroftAbility):
    def __init__(self):
        MycroftAbility.__init__(self)
    
    # Mycroft ought to name this operate instantly when the person 
    # asks to create a brand new merchandise
    def create_item_on_list(self, message):
        move
    
                # Mycroft must also name this operate instantly
    def create_shopping_list(self, message):
        move
    
    # This is just not known as instantly, however as an alternative must be triggered
    # as a part of context conscious selections
    def handle_dont_create_anyways_context(self):
        move
                
    # This operate can be a part of the context conscious determination tree
    def handle_create_anyways_context(self):
        move
    
    
    def cease(self):
        move

The __init__ and initialize strategies

A talent has a couple of “special” capabilities that you must learn about. The __init__(self) methodology is named when the talent is first instantiated. In Python IDEs, variables which are declared exterior of the __init__ part will usually trigger warnings. Therefore, they’re usually used to declare variables or carry out setup actions. However, whilst you can declare variables supposed to match the abilities settings file (extra on this later), you can not use the Mycroft strategies (corresponding to self.settings.get) to retrieve the values. It is usually not acceptable to try to make connections to the surface world from __init__. Also, the __init__ operate is taken into account elective inside Mycroft. Most expertise decide to have one, and it’s thought of the “Pythonic” means of doing issues.

The initialize methodology is named after the talent is totally constructed and registered with the system. It is used to carry out any remaining setup for the talent, together with accessing talent settings. It is elective, nonetheless, and I opted to create a operate that will get the authentication data. I known as it _create_initial_grocery_connection, if you’re curious and wish to look forward. I’ll revisit these two particular capabilities within the subsequent article after I begin strolling by way of creating the talent code.

Finally, there’s a particular operate known as cease(), which I did not use. The cease methodology is named anytime a person says, “stop.” If you could have a long-running course of or audio playback, this methodology is helpful.

Wrapping up

So you now have the define of what you wish to accomplish. This will certainly develop over time. As you develop your talent, you’ll uncover new performance that your talent would require to work optimally.

Next time, I’ll discuss in regards to the sorts of intents you’ll use, the way to set them up, and the way to cope with common expressions. I am going to additionally discover the concept of conversational contexts, that are used for getting suggestions from the person.

Do you could have any feedback, questions, or issues? Leave a remark, go to me on Twitter @linuxovens, or cease by Mycroft skills chat channels.