Designing Great Conversations
Hello everybody. so, I'm Vittorio Banfi. the CEO and co-founder at Bot Sociaty, before we start want to make an experiment. How many of you have ever received an email that said Vittorio from bot society in the subject? All right, cool all this time people. That's great. so, I have a beautiful clicker here. so, I'm just going to click looking at Dustin intensely. I'm the CEO of bot society. BotSociaty is a conversation design solution, we help your creative team and product team to imagine, iterate and design your conversational experience and doesn't really matter whether it's on voice or chat. We got you cover there. and we're the biggest platform out there for conversational design collectively. Our users have design 3.2 million messages. We our biggest investors are Googling 500 startups. That’s some of our clients and we also, have an educational program with more than 80 80 universities 2,000 students, but that's not what I'm talking about today. It's what I want talk about today is why you should design your part or generally speaking your conversational experience and I'm going to tell you some reasons and I'm going tell you the ways to spot whether you actually need to design a conversation interface or not. so, if you're a manager that's another way of asking the same question, right? It's like why should I invest in conversation design, so, I came up with these three patterns that we're seeing in our experience working with a lot of companies, so, the first one is the user experience and then I'm going talk about the engineering optimization and natural language processing optimization. so, let's talk about user experience. Right? So, if you take the stance of if you take the point of view of the of the user was a problem or the user who wants to buy a product from you this the choices that the user will have can go on your website you can download You're a poor go on your mobile website, or you can call up your ivr system most probably right? And so, when you're building a bot, you're adding another interface there, which is going to be here. What happens is you're giving your user different doors. That's one way to visualize that right different doors to interact with you and each of the previous tours had of design team that worked on it that refined it that iterated that user. Experienced a lot and then you come up with a bottle of us interface that's not be designed and it will show so, you're basically presenting your users with this choice and your gaming it, right? so, users you’re the user expectations that you're setting up. Our that each interface is beautifully designed and so, usually the way to spot this is that you have a you see a lot of frustrations from the users using your conversation. Faces and or a general lack of usage and that's usually because of the four doors experiment that you're running you're basically running on intelligence experiment to your users is like, do you are you choosing the well design profitably thoughtfully selected interface or the other one. so, that's the thing that you want to avoid. Because those are the user expectations that have been created. so, you want to dedicate the same design careful design to the new interface. so, this was user expectations engineering. Emulation what I mean by that so, and I'm sure there's developers in the room when you're trying to solve a problem from an engineering perspective. You want the problem to be crisply and detailed when you approach it. You don't want to have a Loosely defined problem. so, and the key word here is in detail. so, what I mean by that is if you want to keep from a management point of view now is if you want to keep your engineering team on track and use the talent that you have need to crisply define the problem, and so, let me make an example here. so, a Loosely defined problem is something like this. We want the users to purchase the products. We want to add stuff to the shopping list and confirm it later. There's an example right now. This is not very well defined. so, what will happen is that your engineering team will either have to fill in the gaps with their own decisions about how the user experience should be or they should they We'll get back to you and ask you a lot of questions. so, this is a longer slide and you don't need to read it all of it. But the idea is you want to crisply define the problem for your engineering team on a level of detail. That is good enough for me as an engineer to come up with the technical solution in the way that you expected to be. Right. so, in this example following here is like series of questions authenticate them that that will be the first step of a purchase process right as I want to know the customer is for example If they don't have a credit card, then we asked for it. And if they have then we proceed, Etc. So, you want to arrive to A definition of the problem, which is detailed enough. And so, how do you spot if this problem it's usually frustration on both sides. so, business is frustrated because the actually like al so, to talk without microphone and the engineering team is frustrated because he continuously have to interrupt the work and go back to business and say stuff like what happens if the user says this what happens is the user says that maybe some of you have experienced that right. so, like all the edge cases are not defined and so, the progress is not showing so, this is engineering frustration. That's not or optimization. Right then if you have been working with Bots, you have seen that you al so, have the natural language processing section, which is kind of separate from the engineering part in that. Okay, you have your framework the body is rolling and then you need to train your natural language processing instance, right? And so, it basically just a quick primer. That's how you Can look at a bot structure regardless of the technology that you pick right? You have a channel like say the Google assistant or the SMS. You have natural language processing engine like rasa or dialogue flow and then you have a dialogue manager. You can use rasa for that as well. It's amazing, right? Okay. so, what happens is let's focus on the NLP part. so, basically the NLP Art is something like this? Right? so, like you have a list of intense and then it's like that's how we express this intense. That's one utterance or turns One turns to right and then you have income to with ezel is on alternatives. so, what happens is if you don't design your bot, you actually don't have a very good map of all the intense that you want to have in your bot. so, basically what happens is you release with this first set of In this example, there are three right then you go on production the you acts kicks in and what that means is since you didn't design this carefully, the user expectations will create the need for a new intent. so, basically the users will keep asking about something that you didn't anticipate because you didn't preview it with you in Prototype. You didn't go through the design iteration, right? And so, what happens at that point? Is that you were going to had another intent and that's where the thing gets complicated because the problem with natural language processing is that now we don't need only to teach about the new utterances of intent for but you will probably al so, need to figure out how to not trigger intent 3 by mistake or intend to in other words the moment that you are. Additional intent you always run the risk of having to retrain part of the previous ones in order to make sure that the new intent is identified correctly. So, for example, let's say they're one intent is new booking. so, it allows you to buy a new plane ticket and in time the new intent that you're adding is changing a booking now, you need to figure out the difference between those two and you need to retrain your MLP again. so, basically, you're losing a lot of time in LP training and this is not due to your data scientist not working correctly. It's just because you didn't design the map of your intense in a detailed enough way to allow them to map all of them previously. And so, these was NLP training optimization. so, those are three main areas that we've seen where that basically allow you if you actually design Bots to meet this expectation meet the user experience expectation in terms of how good and experiences mean the engineering expectation in that once they have a design file that can refer to Detailed enough so, they can provide and go fast in their engineering efforts and on the natural language processing side. Once you have a complete map of all the intense, you don't need to go back and retrain them on the fly up the production kicks in because the map is complete enough. And that's about it. Thank you slide. Thank you. Thank you. Thank you, Vittorio. We're going to do couple of things. so, we have little time I feel a little generous and I want to give away these Echo dots so, I'm going to give three tickets per question. We're going do three questions. questions one two and three when deploying a new chat Bots like you said there's going be a lot of intent that that you are able to think about. How could you minimize the impact to the user when you don't have everything, they're going to be asking about but at the same time you don't want to get them frustrated because the chat bot keeps apologizing for not knowing that information. Yeah, that's a great question. So, one of the things that we see it's very successful as you iterate on the design before actually building the intent structure, right? so, if you schedule a phase where instead of thinking about intense you just focus on the user experience and you create an initial design and You show it to potential users and you see what they ask about and which expectation that type of question and personality creates. Then you can minimize the amount of I would say unexpected intense once you move on to production and if you think about it, you're already doing that in the website and mobile app world. Right? so, like when you're releasing a new mobile app or a website, you don't actually just Russian code it unless you're a one-man band in his kitchen. Okay, that's different. But like if you're a team then you just don't come up with a website, right? Because then the reason the problem is you release it and then people don't click on the button and then you need to rework the website, right? so, you create a design you show it to other people in the company you show it to potential users and then you minimize the risk of people not click on the button. So, if you take the same approach the same workflow and you just flip It for the conversation design world. That's a very good way of avoiding unintended or unexpected intense in production. I hope answer the question. Thank you. so, when you are designing a conversational design for a particular Channel when you're designing a conversation designed for a particular Channel, then actually you might have done the NLP optimization for that particular Channel when are extending that to another similar kind of message kind of channel there any best practices or anything? Actually, your tool will provide. I’m sorry. Not sure I got a question. so, let's practice about the channel. Two questions, which one is actually thing is you talked about NLP optimization you mentioned about NLP optimization. What are the co tool will you are you will provide yes similar to the way for the condo association of design. Do you provide any support for optimizing conditional design for on the NLP side? Yeah. Yeah. so, for example what you can do with but society is you create your design you mop your NLP without still moving on to the building mode. so, you Are free to change it and very fast way like you can drag and drop right and then we have very powerful Integrations with any NLP engine that your engineering team will end up using so, we work with the rasa team with the dialogue flow team. We're going to announce a partnership with the Microsoft but framework team, and so, basically then you take this design data and you transfer it into your NLP. so, you already have the structure and you just need to train it a little bit more on the technology that you pick but you don't have to copy paste everything again. so, do you provide any kind of a mock testing saying that clear this one actually with Facebook Messenger? How will work suppose if I wanted whatever the constant design I have done for Facebook Messenger. I wanted to extend it to I wanted to make it available in slack. Will we be able to use the same optimization or any right? Yeah, you can use you can you can use two different now? Got the question. Sorry. I didn't get it before so, you can use two different workflows. so, the first workflow is use exactly same interaction say that you built on messenger you use it on slack. That's one thing that you can do. That's okay. Another thing that you can do with Bots has ideas you convert the format from messenger to slack and we help you do that with a conversion feature. so, basically have side to side the two different experiences and you can pick so, you can leverage like eh platform capabilities. so, for example, like in slack you can have multiple people talking or you can have a different button set of buttons in this kind of stuff. And so, you basically convert it to the new format and then you can export it again. On your NLP engine while keeping all the other things. so, you basically just change the part of the conversation or should say I should say adjust the part of the conversations that need to be adjusted given the new channel. Sure. Okay question is, does your conversation change communication, you know channel to channel. He was asking like it does an Enterprise of let's say you're designing a conversation and you designing it for us like and Facebook Messenger. Do you find that the design is slightly different for an Enterprise customer versus you know, like a regular customer consumer on Facebook? So, do you actually tweak your design?
So, the conversation is different and question how many people how many people do you typically have in your focus group when you're designing a conversation? conversation? Okay, I'm sorry. There’s so, don't do questions. That's great. Yeah, so, we see both approaches. so, we see teams coming up with one interaction and then just adjusting it for a different channel of let's say from his flock to Messenger right and that's usually the case where you're basically building slack about for a prosper segment. So, not internally, but externally and so, it's an external bottom. Like it's an external bottom messenger is the same use case. It just tweaks it right then we see another type of bots which are kind of like the internal bots. so, they're not in the open there so, that usually the design changes dramatically when the on the use case, right? so, if switching from slack to messenger means that you need to change the use case then in that case, of course, you need to redesign the only interaction just because they use cases is different. We also, see another thing where we offer generic Channel which is kind of like an obstruction of full different channels. And so, we see a lot of teams starting their design their I should not raise my hands because it's been sweating. They say see them. Being there and then they kind of once they have this generic abstract interaction, then they adjusted for his channel. so, they say, okay now let's adjust for Messenger for our website for our mobile app like this in terms of focus groups. It really depends on the type of use case and I can say that at least so, let me say a little bit different. so, if you're running user testing user testing, you can have a small number like 5 or 10 people but qualitative user testing implies that you're spending some time with them. You're asking some question on the end of the interaction. so, you're showing them basically out like one of our customers is an airline and they're lucky enough that their office is the airport and so, they just step out and they were showing the interaction to users. so, 10 users were enough because they could see how they were interacting if it's remote then you get less data out of each. so, we see at least 30 or 35 folks there if to test one use case, so, if you have done multiple use case, then you multiply let's give applause a round of we Tori Thank you very much.