Transcript:
This is the last Talk of the afternoon. You guys already had a whole day of learning about chatbots voice vomits all the technology all the tools. But really, we're going to wrap it up today and really figure and really focus on how to go from an idea zero to launch in the most efficient way possible. I think one thing probably all of us have seen up here is large Enterprises. They have an idea they want to do something. That's a very good use case, but it gets caught up in Pin may be bureaucracy or trying to pick Technologies and they fail to launch or takes them something that could have been a month or two months takes them a year to launch and we've all probably seen that. so, the focus of today's talk is going to be like how to go from zero to launch and I'm going to stand here. Hopefully, I don't fall off the stage. Okay, so, we're going to do a quick intro of everybody on the panel and then we'll get into the questions. Rob blue bow with Bots copy
Johan with Oracle. You kill them all with Bob mock David Raj CEO and co-founder of Passage AI Sergei Bork of co-founder and CEO of Altera. AI Tina Wang co-founder and CTO of transported Okay, so, to go back to the first question and really with the essence of this talk is like how do you go from an idea to an MVP? And what are like the first steps that you have to take before you can really start picking technology or building anything like what are the first steps to go from an idea to an MVP? I can go first. I think it starts with the use case could be customer service IT Help Desk conversational Commerce. so, Identifying the use case and essentially, you know on the use case has to be conversational by definition and then, you know getting all the content you need to respond to the user or the customer and then typically in our industry. We recommend starting with something really simple like an order tracking chatbot. so, you start really simple see how people use it. You might find that customers. Are asking, you know different types of questions in which case you can expand the chatbot account data. I think really quickly sometimes when we build a chatbot especially early on in the early years. It was a solution in search of a problem build a chatbot. If it's the only thing that will solve the problem if some other kind of app or widget or web functionality can solve it great if that doesn't work and it escalates to a human and the human could solve it and you're happy with that great. But if you if you think a chatbot can slide in there and solve Problem. That's
when you build it and not before? so, for us it's kind of little bit different because we have both a platform to help built chatbot. so, we have a platform but al so, we ship a bunch of out-of-the-box chatbot for HCM for human Capital Management for services and things like that, but we work with a lot of customers to kind of take and built the chapel what we typically start with them is to identify the use cases a lot of times their focus areas. For example, one of the most Common use cases is for our service customers. so, a customer contact center wants to divert some of the calls to chop our digital system. So, we direct them to okay, look at these common use cases that could are good for offloading these live chats. And then what we do is we take each one of those customers that would journey of what we call cop. Asian design Workshop. so, each one of them we went through. Okay were the key personas were the key business problem. They're trying to solve and then identify and create a conversation flow based on that. so, for you know, so, for our perspective, it's not so, much. We build a chatbot based on that that we help our customer going through that process to build their chatbot to a fruition. Okay. so, a lot of you probably in your Enterprise you guys have designers’ developers, ux people, copywriters. You have resources to hire new people. so, when you guys work with clients or you guys built Bots, like who do you pull in from other departments? How do you how do you identify some of these skills or if you're going to hire somebody like what types of people are you looking for? so, conversation design is very different area. Right? so, it's you can just say okay. Yeah graphical user interface designer suddenly become a conversation at designer. What we do actually is again with our customer. The starting point is always the line business users who knows their customers the best right you start with them and we have them to help you design a conversation now that may not be the final conversation, but it's No flow, then what you want to do is bring in a conversation designer and this we do have now a conversational user interface designer. I think some of them were in last year's conference to really provide the best practices on how to handle for example, there's an earlier question on how do you handle? You know, when you start with a chatbot have a very small number of intend you can handle how do we handle that? Well, the solution is you best practices start with building an intent for unresolved intent. Your start with that so, there's a lot of these use cases that these best practices that a conversation designer can help to actually in our customers all we're starting to see are these conversations? How should I say a conversation developer and we're conversation architect that are emerging in these customers? And so, we're really seeing them working in tandem with business users to them. with each other so, boo boo boo boo. Yeah, so, I think the first the first person should be the strong product manager somebody who will understand what you are building and basically formulate task and if it is a startup that typically it's yielded. The CEO the founder of the startup is typically the product manager. You don't need a dedicated separate a product manager. if it is inside the company that it should be somebody who kind of takes charge of the boat will be the boat father. so, this is the first most important person and he or she should al so, figure out where to get training data and what how to sell it to the clients and whatnot. And the second important person is the engineer and so, if we are talking about both wizard, some brains not just idea are like buttons, but you want to use some form of machine learning even level 2, then you need somebody who knows machine learning you need you need the strong engineer. Well ideally from Google or something like that and all this machine learning tools that have been open sourced, are still pretty finicky and require skill to use them and train them and so, you need like a PhD in computer science or Google or you know people from Facebook AIA and so, that's two most important person. I think it starts with the use case again. so, let's say the use case is customer service. I would start with talking to folks in the customer service team the head customer service identifies the top. Let's say ten issues that you know account for most of your incoming calls if I start with that and then the second, you know would be to hire machine learning engineers. Was the accuracy and the NLP the is really important. If you have a chat bot that is not accurate response incorrectly to the customers questions. It's actually going to be counterproductive. so, I would start with hiring machine learning Engineers full stack Engineers. If you have to connect to different platforms and finally conversation designers that's are really key aspect of the chat box. One inside YP help for there's a lot of out-of-work copy writers who have 10 20 years of experience in they're just phenomenal writers’ real pros and you could get them cheap and find the ones that are really geeky tech savvy as well. So, they're copywriters but their science geeks on the side sit the next to somebody who knows how to use dialogue flow. so, this person is less creative, but they're more you X oriented and can actually do a little bit of development. And you that team those two people can make a really great chatbot if they go to chat Bots life and they look at the articles there follow the steps. They can get into a lot of trouble in a good way. Yeah, I'll just add a little piece there that we talk a lot about conversational designers and you know actually chat Bots are new but things like SMS and voice are not and so, definitely look for people who have skill set designing old SMS interfaces that can often be a good place to find that kind of resource. I'll add very quickly here that I think but on the side of the product manager, it needs to be being able to split up those like different roles as well write with Voice and text and being able to recognize those things ideally with the help of a conversation designer. so, even for a certain use case, I think bringing it all together is really the key thing and a product person can usually start that off in conversation was that I can take that then the engineers you can sort of put on that down a few different ways. But yeah. Okay, great. Great sits like on building your team. so, you have your initial idea you have your team. so, now how do you actually get it to where you start designing the conversations and then take it one step further to actually build something and get it live. Like what tools are what Frameworks do use to start designing the conversations and then al so, get it live. so, we actually did a lot of this we would make these Maps dust and I when we were presenting, we showed a quick glimpse of a very small map. But anything that you can use to draw boxes and lines and arrows start mapping out the architecture at that time, we were using Euro which was real time board, but now technology is like, you know about society and Bot Mark are fantastic for kind of mapping out the beginnings of a bot. Yeah, I think taking the step before really. Into anything before like you really got to line up your testing line up like a term of conceptual Frameworks. I think like validation as you would with many other MVPs or products that you're building is super key. And obviously, I think the first step is like the mapping pardon and even after that being able to identify the best like metrics that you're going to go through and identifying that does depend on the use case, of course, but being able to put that on paper being able to al so, visualize that the best you can and see what you can do before you even start building before you touch a line of code is a very crucial. She'll step in this process. You have to know your kpis. Yeah for slows towards those kpis absolutely doubt that you're blind. I think as with any product you start with the customer and work backwards. so, you identify the customer problem and then find a technology solution that solves that problem. so, it starts with that and then typically what we recommend is you prototype the bot before you add machine learning it can be all canned and you know robotic but if you prototype it then you can get users. To actually use it and give you feedback. Is it really useful? Is it solving a problem and you can use you know a number of tools for that and thirdly it you know, once you have launched it in our look at the metrics look at the analytics look at customer satisfaction and then iterate a lot, you know, improve the machine learning algorithms add more intents as people start using it, so, you know iterations are really important part of what building. Our yes, I agree to relations are very important. so, you prepare your training Corpus your kind of intense standard answers and you think that you thought of everything but of course you would be all always wrong that the reality it never fits your plans. so, half of the intense that you define will not be used people don't give a damn about them. However, the other you know, there will be another whatever hundred intense that you didn't think about at the planning stage. So, you have to check the chat logs and see what people is actually asking and quickly update your Corpus training set everything use cases to be able to put the board to answer the questions that are frequently Asked not those. Those that you thought would be important. Yeah, I think that you know, I'm a little biased because at Trends Posit, we build a platform to make it easier to talk to apis but I think going back to this message being able to quickly iterate and prototype on the functionality of the bot somewhat independent of the machine learning around conversational understanding is how you can sort of separate out those two pieces because I think there's the can you understand what someone's saying to the bot but then al so, like is your boss actually able to do useful tasks that Real value and the faster you can get a prototype out there and get metrics and feedback the better. Okay, great tips. so, we skipped ahead a little bit on this question and I think a lot of us already do this internally when we were excited about a project and we want to get a launch and we see this a lot is how do you get this through compliance? How you how do you deal with security? How do you deal with gdpr? How you deal with the concerns from BPS about customer data and conversation history and stuff like that. You guys have any tips for navigating that kind of internal? Clients. Yeah, I do. I think having approval workflow is super powerful, especially these with all these things coming up. I think that that is it's almost basically key. Now, I think to have some sort of process where you can even in that iteration process even in that very early process thinking about those things earlier than later is probably the safer bet today. so, I see that as you know, that's each company will develop its own each use case will develop its own but just like standardizing that throughout everybody who's involved after you have that team ready. Can be a really crucial piece here. so, I'm going to be maybe a little biased but I shouldn't say that but you probably want to start with a platform that's certified with that has a clear stand around for example gdpr and fed ramp and different standard and al so, the other things to realize is that while the platform give you a path to say how to be compliant with you to give you the tool to be GDP are compliant. For example, you still have to design your Digital system Rabat to be digital GDP are compliant. so, there is you know, so, it doesn't completely remove the need to understand what those requirements are. But if you looking for let's say, you know, we're in platform business someone's looking for a platform will would help them to do is guide them through the process of what those issues are and making sure that when they designed their chatbot, they understand what facilities are in the platform to help them to be GDP are compliant. Yes, you can remove move or give the user option to remove conversation history. How do then deal with analytics and things like that? so, I think it starts with data privacy when you're dealing with personally identifiable information, you know that data has to be completely secure. Otherwise, you lose the trust of the customer. so, for example, if you take a use case like a loan application and you ask for a person's social security number clearly that has to be kept extremely secure and private there are you know compliance. You know things like gdpr which are all part of this another one from a regulation perspective is you know, in situations like healthcare or even you know, while you're driving talking to the car and saying the check engine light is on and the chat bot tells you what the problem could be. Their accuracy is really important and maybe disclaimers are fairly important, you know. No one use case in healthcare is diagnosing a problem by asking you your symptoms clearly that has to perform at a hundred percent accuracy and you need have disclaimer saying it could be this condition. Why don't you make an appointment with a doctor thing like that? Okay, this might be useful unlike old school pieces of marketing. You can't just hand it off to the lawyers and say approve this so, you can't just hand it off. Bots are very complex. Their labyrinths are of content and they need to change on the Fly very quickly. so, how do you do that? We were talking to some huge Banks and they were running into issues about how do we approve every single response with our lawyers? It takes forever and you need to be fluid and Nimble if you want to have a good bot. So, you know when you think a Service people not every word. They say is approved by lawyers. You know, you talked to support person at Apple their kind of winging it they might throw out an analogy or something to help you because they're authorized to do that. so, your chatbot writers should be authorized to use their discretion within certain confines, but maybe let go of this idea of approving every single letter because it's just not tenable. Very good points, very good points. Okay. so, now your boat's live describe kind of how your iteration process of looking at analytics how to improve the Bots over time. What does that kind of look like for you guys like in the first months of the body being life? so, again, I'll just speak from a platform perspective. One of the things that we always tell our customer is you start with, you know, a limited limit the number of intent and then roll it out to the customer to users and then use the analytics features that look at for example are the top on re solve intent wear a sari top on re solve utterances and use the whatever facility obviously work-life a platform of the facility, but You know, there's a lot of analytics companies buying a little Scamp can also, help you with that which is identify which are these top unresolved utterances and then allow human to train to say, okay. This utterance actually maps to that particular intent, but something we do al so, kind of recommend is not letting the chat bar to self-learn because it could easily be abused or beam is trained by unintended user. So, always use a supervised learning and human learning to look at these analytics data that literally trained the digital assistant adding more and more utterances to each of the intent and iterate through that. I'll add something quickly. They're like, I totally agree with that. I think that with convert like conversation design never stops. Basically like the mold like when you see those analytics when you see those messages coming in where the most frustrations are where the most confusion is like you got to keep iterating on that process and I think the maintenance of conversation design is becoming ever more important now as projects are expanding as they use cases are getting more serious like Beyond just you know, simple customer support thing, especially if you're going from a those Morecambe like complex things. You know considering that you might have to iterate a little bit even after the initial piece is something that you just have to like move forward with and realize how crucial that can be in the long run for satisfaction and utility as well. Yeah, definitely agree that analytics are really important you get to see how users are using it. For example, we report sentiment of every message and the average sentiment of you know of users that day and that can be used as a proxy for customer satisfaction. so, looking at how the board is being used seeing messages that went to a live agent because that will give your ideas to further automate using AI. And then the customer satisfaction piece and mention and then when the bot launched, it's not going to be a hundred percent accurate. There will be some NLP accuracy issues, but that could actually be a teachable moment or a trainable moment. If you will where you can actually take that those exact same messages label them and use that as training data and improve the Bots for them. And as we talked all about measuring analytics for your chatbots, I think is al so, important that chat Bots don't live in isolation of your greater business ecosystem. so, make sure you get that whatever kpi that your boss is really trying to address that you al so, tracking that metric as well to see the validity of your boss. Yes, so, unlike the website or mobile app the chat Bots. Still need to be developed after lunch you because it's well if it has machine learning if it is not just a they are if it if it has machine learning then it has to learn from new. Post from new data points and the chat box. Don't do it banned by themselves. You need somebody who will teach the bird so, you need the person who teaches the board for a long time after lunch. You have to be ready for that. Okay, so, I think that wraps up our talk we touch from how to conceptualize how to go live. Hopefully that's very useful for you guys and you can bring that into your Enterprise and you don't spend years trying to get an idea out there and I think we're going to go to questions. Hi, so, you have mentioned about authorization. I actually didn't follow is it like restricted to the backend system which we integrate or can we do something at the language level? What are your thoughts on that? the authorization, so, I just wanted to make sure we understand the question. so, basically authorization in terms of connecting Chapa with the Enterprise security is a kind of a what yeah, so, actually that's pretty important. Actually that's one of the key the first requirement these for us is to be able to integrate a chatbot security with Enterprise security, but the Enterprise that but typically the authorization and user would need to come from the backend application because what Want to make sure is the chat pod can access the same the user who acts usually chat about to access information as the same access level as the actual user the backend system because Enterprise application has data access controls and things like that. so, it's important to be able to integrate and leverage. They use the backend application security for your chat bot to prevent having, you know chat by user can see all of the data whereas that users actually authorized just to see part of data. I hope that's the question.
Yeah, if it's about authentication like logging in a user and then providing personal information, there's actually a feature that most messaging platform Support called account linking so, you can use that feature and use the customers odd system or authentication system to login the user. so, one of the important things while building the child bolt is building these stories, right? so, when we Build a chatbot, sometimes every story is not fully developed after 6 months. Maybe eight months. We figure out we have a hundred more user story. But what happened with the previous 40 this story when we change the intent or entirety, you know, so, how we take how we can take care of such a painful problem that we have to go each and every user story to change the intensity.
Is there any better way to take care of this? I think having a good platform that you can call your central source of Truth is very much key here. I think there's definitely risks with trying to spread out too much information like with teams or has to be a separation of what like, you know, what's using and some of the perspectives but at the same time it should be there should be a careful process of figuring out the right ways to make sure that the stories are consistent by really holding it like in one in one Central spot in sort of continuously like iterating on even that that piece of it with can save a lot of time and just making sure that all that comes together. about dialogue management and with so, many tools out there like dialog flow and rassa. Is there still an argent to be made for a custom-built application for dialogue management?
Yeah, like wouldn't want anybody still write the code to manage the dialogue outside of the nlu to figure out like what the response is going to be or would you recommend that everyone just uses something out of the box like rassa or dialogue flow? Well, I probably have a contrarian view but I believe that all this dialogue flows suck and to Gold make just a glorified idea that only natural language understanding is the way to go. Everything else has is a they are in this case. So, this is basically making the point moot. However, if you want to do some dialogue management, it’s easy to code manually, but you don't need all these tools because it's kind of simple State machine, you know, any second whatever so, far more computer science. You don't can't write a program of this whatever 20 steps. You don't need the tool for this the real the real difficult part is natural language understanding. and so, I think to your point there's definitely a place where think dialogue Builder and dialogue management system. However, what we see is it's a great place to start right? so, when you design interactively work with your user and customer design your champ, I use these dialogue flow designers to prototype and I think boss society was earlier and had their very good tool and we have a tool as well for prototyping that but you al so, want to Tool that can allow you to just totally customize a dialogue experience. Once you build the initial dialogue you want to be able to go back in and essential as a coder to be able to add all the branch and conditions and adding all the other custom logic want to do that. so, my perspective. Yeah, there's definitely place for a lot of the initial designing and creation. But then to really refine it you really need a tool al so, allows you to get to the underlying code to really get the dialogue is Screams like you want it? Thank you very much. Can get a round of applause for the panel zero to lunch?