Tag Archives: Cortana

//Build Reaction – Cortana and the Bot Framework

Microsoft Bot Framework

Bots (or conversation agents) are rapidly becoming an integral part of your users’ digital experience – they are as vital a way for users to interact with a service or application as is a web site or a mobile experience. Bots can also integrate with digital assistants like Cortana. Developers writing bots all face the same problems: bots require basic I/O; they must have language and dialog skills; and they must connect to users – preferably in any conversation experience and language the user chooses.

In this video, you’ll learn the basics on how to build and connect intelligent bots to interact with your users naturally wherever they are, from text/sms to Skype, Slack, Office 365 mail and other popular services. I explore the new Microsoft Bot Framework recently announced at Build 2016, which provides just what you need to build and connect intelligent bots that interact naturally wherever your users are talking.

Through simple demos I cover the Bot Builder SDK with C# (Node.js is also supported), the Bot Framework Emulator and I also explore how to handle natural language input from the user with the Language Understanding Intelligent Service (LUIS) from Microsoft Cognitive Services. Every business needs bots to provide a more personal experience to its users and customers. This video gives you the basics to get started in just 30 minutes, and then points you in the right direction to learn much more.

You can watch the video using the embedded player below or directly on Channel 9 here.

You should also check out my colleague Jennifer Marsman’s blog post and reaction video about machine Learning at Build here.

Additional References – Cognitive Services & LUIS

To learn more about the Microsoft Cognitive Services (aka “Project Oxford”) and the Language Understanding Intelligent Service (LUIS), you can watch the following sessions from Build 2016:

Building a Conversational Bot: From 0 to 60

Microsoft Cognitive Services: Give Your Apps a Human Side

Microsoft Cognitive Services: Build Smarter and More Engaging Experiences

Cortana Extensibility

Cortana – Microsoft’s Digital Personal Assistant for Windows 10, iOS and Android – is evolving from reminding users about things to helping them get things done. Cortana can now proactively surface your app to users in the right context, and provide your app with relevant information to act on, making it quick and easy for the user to work with your app, or even complete tasks on the user’s behalf. You can watch breakout sessions from Build 2016 for a walkthrough of these new Cortana capabilities, and how your existing UWP apps – with minimal effort – can use them to increase engagement and discovery.

Personal Assistants: The New Context-Aware Digital Runtime

Cortana: Learn How Cortana’s New Capabilities can Proactively Drive User Engagement with Your Apps

Cortana Futures: Step-by-step on How to Teach Cortana to Proactively Engage with Your App

If you have any questions, you can post them in the comments section below, or you can contact me on Twitter at @ActiveNick.

Get Ready for Windows 10: Learning Universal Windows App Development

Windows10iscoming

Windows 10 is coming this summer! It’s even already available in preview today. With it comes a huge opportunity for developers to build apps once for the Universal Windows Platform and deploy/publish these apps to Windows phones, phablets, small tablets, full tablets, hybrids, 2-in-1’s, netbooks, ultrabooks, notebooks, laptops, desktops and even servers. You’ll also be able to use the same apps/code when running Windows on IoT devices like the Raspberry Pi 2, and soon enough you’ll be able to publish those same apps on Xbox One! But the main question is:

Are you ready for Windows 10?

The BUILD 2015 conference starts in less than a week and you’ll be able to watch live streams of countless sessions to bring you up to speed on many development topics, from the cloud to mobile development, including IoT and game development.

You probably want to capitalize on this new wave of Windows 10 opportunities, but many of you may not be too familiar with app development for Windows devices. Fortunately there is a way for you to catch-up, and fast. And the best part is it’s completely free. It’s called Microsoft Virtual Academy.

Successful technologists never stop learning and great technology never stops evolving. Microsoft Virtual Academy (MVA) offers online Microsoft training delivered by experts to help technologists continually learn, with hundreds of Microsoft training courses, in 14 different languages. MVA’s mission is to help developers, knowledgeable IT professionals and advanced students learn the latest technology, build their skills, and advance their careers.

With hundreds of free courses in MVA, it can be quite a daunting task to get started. Which course should you take? This blog post is your learning guide, providing you with the roadmap you need to learn Universal Windows App development, and get ready for Windows 10.

I’ve outlined the recommended courses based on your current skill level. Feel free to dive in at the level that is most appropriate for you:

  • Absolute Beginners
  • Beginner / Intermediate-level App Developers (Windows Store or Windows Phone)
  • Experienced / Published Mobile App Developers on Windows

To help you get started with the tools you need, make sure to read my Ultimate Guide & FAQ to Setting-up a Windows Phone Development Environment.

ABSOLUTE BEGINNERS

Start with this section if you’re completely new to programming, or if you are completely new to app development in general.

C# Fundamentals for Absolute Beginners
Want to learn a different language? Over the course of 25 episodes, our friend Bob Tabor, from www.LearnVisualStudio.net, teaches you the fundamentals of visual C# programming. Tune in to learn C# concepts applicable to video games, mobile environments, and client applications. We walk you through getting the tools, writing code, debugging features, customizations, and much more! Each concept in this C# for beginners course is broken into its own video so you can search for and focus on the information you need.

Introduction to Mobile App Development
Nothing motivates students more than building an app that they care about and that they can immediately see, use, and share. Get insights from Windows Platform Developer MVP​ Lance McCarthy, as he teaches students to build a mobile app using Windows App Studio and to then extend and enhance the app using Visual Studio. Students learn the basics of the app ecosystem and Software Development Lifecycle. They also learn about code modification and additional basic app coding skills, including the topics of variables, simple data types, conditional programming constructs, and simple library classes. This course addresses AP Computer Science learning requirements.

Windows Phone 8.1 Development for Absolute Beginners
Join Bob Tabor (LearnVisualStudio.NET) for this 9+ hour series as he covers Windows Phone UI with XAML layout and events, navigation model, application lifecycle, and working with the Windows Phone Emulator. This course focuses on Windows Phone development for beginners as Tabor explores understanding XAML, MVVM (Model-View-ViewModel) and HTML apps in the WebView. Get details about storage, maps, animations, and media (video/audio with the MediaElement control). Build five apps, covering a range of scenarios, from media playback to hosted HTML, from accessing geolocation data and mapping to extending your Windows Phone app to become a universal Windows/Windows Phone app. These Windows Phone development tutorials will build a firm foundation for your future in mobile app development.

BEGINNER / INTERMEDIATE-LEVEL WINDOWS PHONE or WINDOWS STORE DEVELOPERS

Start with this section if you are an experienced developer but new to app development on Windows, or if you have some experience with Windows Phone and/or Windows Store development but your knowledge is spotty.

Building Apps for Windows Phone 8.1 Jump Start
If you’re an app developer who wants to design and build apps for Windows Phone 8.1 using XAML and C#, check out this two-and-a-half day, on-demand course, taught by experts with years of experience developing—and writing about the process. The sessions focus on building apps for Windows Phone 8.1 in Visual Studio and creating universal app projects that share a high percentage of code and that target both Windows and Windows Phone. Get tips and tricks on maximizing your app compatibility and optimizing your code. Find out about new features, and learn how to program the many new Windows Runtime APIs that are available to both Windows Store apps and to Windows Silverlight apps. You can even get code samples. Want to build an app? Watch this course and make your apps for Windows Phone world-ready!

Developing Universal Windows Apps with C# and XAML
Get real-world guidance for developing creating universal Windows apps, and save yourself valuable time when creating developing an apps for today’s mobile workforce and consumer marketplace. Learn from Microsoft experts as they build a working app using XAML and C# development tools and techniques that can give you a dramatic advantage as a developer when targeting both Windows and Windows Phone devices. See what’s smart to share and what’s not, when developing for the two platforms. Explore a broad range of features, covering both consumer and enterprise scenarios. Jerry Nixon and Daren May bring together best practices and key insights from Microsoft internal teams, including the built-in code-generation tools in Visual Studio that can automatically build out hundreds of classes and thousands of lines of code.

EXPERIENCED / PUBLISHED MOBILE DEVELOPERS ON WINDOWS

So you’ve published some apps already and you think you’re an expert? I bet you can still learn a thing or two. Check out these MVA courses and feel free to pick & choose the individual modules that interest you the most. In any case, make sure to watch the last MVA course listed here: A Developer’s Guide to Windows 10 Preview.

Azure Mobile Services and API Management
Want your business to compete in a mobile first, cloud first world? Microsoft Azure can help. For example, Turnkey API Management capabilities help you share APIs with partners securely, and Mobile Services help you to build enterprise-grade mobile experiences for consumers and employees alike, in record time. Get the details and helpful tips from the experts, in this practical course.

Windows 8.1 Developer Training: Geek Edition Jump Start
HTML and XAML developers, are you ready to start having fun with Windows 8.1? Join Microsoft experts for an on-demand version of the highly successful dive deep into the gadget and devices side of Windows 8.1. Explore 3D printing, the LEGO EV3 platform (and how to use it from Bluetooth and USB), how to get speech out of your modern app, and more. See some exciting demos, and find out about awesome new features in both HTML and XAML. Build on your core skills, take advantage of everything Windows 8.1 has to offer, and dive into the Internet of Things (IoT).

While the whole course is definitely interesting, allow me to call out these specific modules to round-out your Windows app development knowledge:

  • Module 4 – Speech, Camera and Microphone
  • Module 7 – Bluetooth Overview
  • Module 10 – Performance

Universal Windows App Development with Cortana and the Speech SDK
Want to add Cortana to your app? Whether you’re into speech integration technology for accessibility, social responsibility, or gaming (or all of the above!), you’re probably excited about the possibilities that Cortana offers for Windows Phone developers today and Windows 10 app developers in the near future. Mobility pioneer Nick Landry and popular author/teacher Jeremy Foster share their practical experience in computer speech technologies and mobile app development scenarios. Explore the why and how of speech apps, tour Cortana, and review the capabilities of the Speech SDK in Windows and Windows Phone. Work with speech synthesis, look at integrating Cortana into your app, check out voice commands, and find out how to build speech-enabled mobile apps with Visual Studio for Windows devices. You’d be surprised at what you can do with as little as three lines of code!

A Developer’s Guide to Windows 10
Want an in-depth look at the Universal Windows Platform? Engaging experts Jerry Nixon and Andy Wigley return to show you how Windows capabilities and social integration can help you create amazing experiences on devices running Windows 10, including phones, tablets, PCs, and even (coming soon) Xbox! Jerry and Andy host three demo-rich days of Windows 10 goodness, explore cool new features, and offer practical guidance. Each session lasts about 30 minutes, so you can join us for the topics that interest you most.

Are you already one of 3 million students who cannot get enough of Microsoft Virtual Academy? What are your favorite courses? Who are your favorite instructors? Are there other courses you would like to recommend to your fellow students in addition to this list? Make sure to let me know in the comments below, or contact me on Twitter at @ActiveNick.

CortanaMVAPresenters

Event Session – Beyond Cortana & Siri: Using Speech Recognition & Speech Synthesis for the Next Generation of Mobile Apps

Cortana Halo 4 HD

by Nick Landry

Speech is probably the topic I’m most passionate about when it comes to app development (ok, I have a soft spot for GIS too). From HAL9000 in 2001: A Space Odyssey and Joshua in WarGames, to Star Trek computers, Siri and Cortana, having conversations with a semi-sentient computer using natural language and speech is probably the ultimate frontier of technology. But speech can also be a responsibility for us developers to make sure our apps are usable by all, and to keep our users – and those around them – safe. This talk is one of my favorite. It’s about using Speech Recognition & Speech Synthesis to build the next generation of mobile apps.

I recently presented this talk at Philly Code Camp 2014 last weekend, and at the Microsoft Mobile App Devs of New Jersey (MMAD) Meetup. I’ve also presented it at Internet Week NY 2014 last month, and I’ve done variations of this talk at other events in the past including VSLive, CodePalousa, DevTeach, DVLUP Day Boston and M3 Conference.

Session Description

Our society has a problem. Individuals are hooked on apps, phones, tablets and social networking. We created these devices and these apps that have become a core part of our lives but we stopped short. We failed to recognize some of the problematic situations where our apps are used. People are texting, emailing and chatting while driving. Pedestrians walk into busy intersections and into sidewalk hazards because they refuse to put their phone down. We cannot entirely blame them. We created a mobile revolution, and now we just can’t simply ask them to put it on hold when it’s not convenient. It’s almost an addiction and too often it has led to fatal results.

Furthermore, mobile applications are not always easy to work with due to the small screen and on-screen keyboard. Other people struggle to use traditional computing devices due to handicaps. Using our voice is a natural form of communication amongst humans. Ever since 2001: A Space Odyssey, we’ve been dreaming of computers who can converse with us like HAL9000 or the Star Trek computers. Or maybe you’re part of the new generation of geeks dreaming of Halo’s Cortana? Thanks to the new advances and SDKs for speech recognition and synthesis (aka text-to-speech), we are now several steps closer to this reality. Siri is not the end game, she’s the beginning.

This session explores the design models and development techniques you can use to add voice recognition to your mobile applications, including in-app commands, standard & custom grammars, and voice commands usable outside your app. We’ll also see how your apps can respond to the user via speech synthesis, opening-up a new world of hands-free scenarios. This reality is here, you’ll see actual live cross-platform demos with speech and you can now learn how to do it. Speech support is not just cool or a convenience, it should be a necessity in many apps.

Session Slides

You can view & download the slides for this talk from my Slideshare account here, or you can use the embedded viewer below.

Session Demos

You can download the demos and samples for this session using the links below:

Session Links and Resources

If you have questions about this session, you can ask them in the comments section below or contact me on Twitter at @ActiveNick. If you’re interested in inviting me to present this talk at your event or meetup, you can reach me via my contact form here.

Nick-SpeechSDK-Talk