First and foremost it's appearing on Samsung's phones, but it is coming to other devices. So here's our guide to Bixby and what it can do.
What is Samsung Bixby?
First announced via a Samsung blog post, Bixby is an artificial intelligence system that's designed to make device interaction easier, specifically designed to avoid the complexity of increasingly fully-featured devices. It made its debut on the Samsung Galaxy S8 and Note 8 devices, but is designed to work across a range of Samsung products including smart fridges, TVs and more.
More Bixby devices
Samsung's Family Hub now has SmartThings and Bixby support. SmartThings is Samsung's own Internet of Things ecosystem that works with thousands of other smart home devices. You can answer your Ring doorbell through the refrigerator screen, for example, seeing who is standing at your doorstep in the process.
Bixby voice control has been added too, so you don't always need to use your hands if, say, they are covered in flour. You can also have the latest news read out, or the weather or calendar entries. And Bixby learns individual voices, so will personalise answers depending on who asks.
Alluded to during CES 2018 in January, Samsung plans to integrate its artificially intelligent assistant in numerous products over the coming year, including TVs and kitchen appliances, the new TVs will be first and the others will follow.
Washing machines, for example, will come with Bixby from early spring.
Enhancements to Bixby
In October 2017, Samsung announced the second-generation version of Bixby. According to the company, Bixby 2.0 is a "fundamental leap forward for digital assistants" and a "bold reinvention of the platform". In a nutshell: Bixby 2.0 is designed to make Bixby available on "any and all devices". It will be available on smartphones, TVs, refrigerators, washers, speakers, and other connected devices.
Bixby 2.0 will be "open", allowing developers to integrate the voice assistant into their products and choose how you will be able to interact with their services. So, we suspect several companies will announce Bixby-powered smart speakers in the coming months. Samsung also said Bixby 2.0 features enhanced natural language capabilities, which will result in more natural commands and complex processing.
Samsung ran into performance issues with Bixby early on and wasn't able to include the English version on the Galaxy S8 and S8 Plus at launch. Its English voice recognition capabilities weren't up to par, and we're guessing Samsung has been working hard to improve it. Going forward, the company said Bixby 2.0 will "really get to know and understand" who you are as well as the members of your family.
Samsung is launching a private beta program with the Bixby SDK. It's now available for select developers to develop with particular third-party apps.
How does Bixby work?
Samsung has confirmed some of the underlying principles of Bixby, detailing that there are three pillars that help drive it:
- Bixby is a complete solution: it is designed to let you carry out a full range of interactions, rather than launching an app for example or carrying out a single task. Samsung says that Bixby will be able to do just about everything you can do with an app using touch.
- Bixby is contextually aware: this is one of the buzzwords of AI, demonstrated by Google Assistant, for example. This will mean that Bixby can recognise the state that the app is in and take the right actions based on your requests, also letting you mix voice or touch.
- Bixby understands natural language: this means that you don't need to use set phrases, but you can give incomplete information and Bixby can interpret and take action. Natural language recognition has been key to the rise of Alexa, for example, and is now a key element of modern AI.
The service essentially works in the same way as other AI solutions like Google Assistant or Amazon Alexa in that it listens to your voice, interprets the information and returns the resulting action. The contextual awareness means you can get it to take actions without laboriously detailing exactly what to do with what - it already understands where you are so can take the logical next step, so there's no need to use the precise language that you did on older voice-control systems.
On the Samsung Galaxy S8, S8+ and Note 8 there is a Bixby button on the left-hand side of the phone. This is used to open the Bixby agent so that you can speak a command, or you can use the hotword "Hi Bixby", once you've set it up. To access Bixby Voice you'll have to press and hold the button and talk, once you release the button, Bixby interprets what it has heard.
For those using the "Hi Bixby" wake word, you'll be able to talk to your device using natural language as you might do with Google Assistant. However, Bixby seems prone to launching accidentally, so using the button press method prevents false recognition.
This only applies to the voice control side of Bixby, otherwise you'll find things like Bixby Vision integrated into the camera and ready to use at a tap.
What Bixby can do
One of the main aims of Bixby according to Samsung, is to deal with increasingly complicated devices. Not only did Samsung say that in its blog post, but every time Samsung demos Bixby or talks about it, it's focusing on device control first and foremost - especially around complex tasks.
Here are some examples of what Bixby will do:
- On saying "show this on my TV", the Galaxy S8 screen can mirrored on a Samsung TV.
- On saying "use this as my wallpaper" the S8 saves the onscreen image as the wallpaper for the phone.
- On saying "take a selfie and share it will Facebook, the phone tries to do exactly that.
These commands all make logical sense, and fitting with the outlined aims that Samsung originally set out for Bixby, it's easy to ask it to do things like changing the volume or increasing the brightness of the phone. Generally, when it comes to device control, Bixby is very good, as it is with composing messages for you, or reading incoming messages from your contacts.
Care needs to be taken, however: the Hi Bixby hotword does trigger easily and we once had the phone erroneously fire up Bixby, where it listened to some of the conversation in the room (about assistants) and then launched the phone's visual disability mode, which took some time to then disable again so we could use the phone as normal.
Bixby Vision offers a range of functions that take advantage of the camera on the phone. Working in a similar way to Google Goggles or the Amazon shopping app you can either ask Bixby what something is, or open the camera app and hit the Bixby Vision button. This is a great feature, but one we've used plenty of times before in other apps.
Bixby Vision will essentially identify whatever the camera is pointed at, with options presented depending on what it sees, offering to identify an image, place, text or go to shopping options. When the Bixby Vision thinks it can see wine, it offers a wine search function. These then lead into more information, once the device has been identified.
We've seen it:
- Identify The Shard in London and suggest local restaurants.
- Recognise a box of Lindt Lindor (strawberries and cream flavour) and suggest shopping options.
- Spot a bottle of Casillero di Diablo and offer details on the wine.
- Recognise text, making it easy to copy text from a poster.
The best bit about Bixby Vision is that it's not dependent on the full Bixby Voice service, with direct access through the camera, so even those who don't yet have language support for the full Bixby Voice service will get Bixby Vision.
There's another side to Bixby. For those not diving into the voice service - or those who can't get it because it's not supported in their country - then there's Bixby Home. This is part of your smartphone home screen, living off to the left of your homepage.
In the UK this integrates the Upday news service (accessed via a left-swipe on the screen or a short press on the button) or Flipboard for those elsewhere and pulls in cards from supported apps. This is very much like the Google Now page, filled with information cards, so it's like a personal feed.
Many apps and services can be integrated into Bixby Home, like Twitter, Facebook, Spotify and your Gallery, so you can access highlights from all these things there. That means you'll have a card showing pictures you've recently taken, a card showing current Twitter trends or controls for your Spotify music.
In reality there's very little that Bixby Home does that's unique or useful: it's just another feeds app like HTC's BlinkFeed and we question whether it will be useful to anyone. Smartphone users are likely to head to the information they want, rather than scrolling through snippets of information scraped from elsewhere on the phone.
You can, however, turn it off easily, so it doesn't appear on your home screen, however it's always there and launches whenever you press and release the Bixby button.
Samsung is planning to expand the reach of Bixby with the launch of a Bixby speaker to compete with Google Home and Amazon Alexa.