Forget about Google Assistant, Samsung's new Android flagships, the Galaxy S8 and the Galaxy S8+ sport their own voice assistant called Bixby.

It's the company's attempt to take on the likes of Siri, Google Assistant, Cortana and Alexa, with a new AI agent that's going to span Samsung devices. First and foremost it's appearing on Samsung's phones, but what is it and what will it do?

Fully announced via a Samsung blog post, Bixby is an artificial intelligence system that's designed to make device interaction easier, specifically designed to avoid the complexity of increasingly fully-featured devices. It will make its debut on the Samsung Galaxy S8 devices, but is designed to work across a range of Samsung products.

In it's launch format on the S8, it encompasses a range of different features, Bixby Voice, Bixby Vision, Bixby Home and Bixby Reminder, for example.

In the future we expect to be using Bixby to control the TV, air conditioning as well as phones and Samsung uses this new service to provide one method of interaction with a full range of technologies.

Samsung has confirmed some of the underlying principles of Bixby, detailing that there are three pillars that help drive it:

  1. Bixby is a complete solution: it is designed to let you carry out a full range of interactions, rather than launching an app for example, or carrying out a single task. Samsung says that Bixby will be able to do just about everything you can do with an app using touch.
  2. Bixby is contextually aware: this is one of the buzzwords of AI, demonstrated by Google Assistant, for example. This will mean that Bixby can recognise the state that the app is in and take the right actions based on your requests, also letting you mix voice or touch.
  3. Bixby understands natural language: this means that you don't need to use set phrases, but you can give incomplete information and Bixby can interpret and take action. Natural language recognition has been key to the rise of Alexa, for example, and is now a key element of modern AI.

The service essentially works in the same way as other AI solutions like Google Assistant or Amazon Alexa in that it listens to your voice, interprets the information and returns the resulting action. The contextual awareness means you can get it to take actions without laboriously detailing exactly what to do with what - it already understands where you are so can take the logical next step.

On the Samsung Galaxy S8 and S8+ there is a Bixby button on the left-hand side of the phone. This is used to open the Bixby agent so that you can speak a command.

At the moment we don't know how much of Bixby is online and how much is offline, i.e., whether you'll need a connection to the internet for Bixby to work in all cases, but we do know it's a cloud based service in the main.

Here's what we know from Samsung and from rumours about what Bixby will do.

One of the main aims of Bixby according to Samsung, is to deal with increasingly complicated devices. Not only did Samsung say that in its blog post, but it repeated exactly the same thing when demoing Bixby to us, saying that devices are now very complicated and a smart AI can save you having to look through the settings.

We've seen this in action too:

  • On saying "show this on my TV", the Galaxy S8 screen was mirrored on a Samsung KS7000.
  • On saying "use this as my wallpaper" the S8 saved the onscreen image as the wallpaper for the phone.
  • Placing a call instantly.

Simple functions, but Bixby saved the need dig those things out, instead only needing a press of the side button and a spoken command.

Pocket-lintwhat is bixby samsung s smart ai explained image 4

Bixby Vision offers a range of functions that take advantage of the camera on the phone. Working in a similar way to Google Goggles or the Amazon shopping app you can either ask Bixby what something is, or open the camera app and hit the Bixby Vision button. This is a great feature, but one we've used plenty of times before in other apps.

Bixby Vision will essentially identify whatever the camera is pointed at, with options presented depending on what it sees, offering to identify an image, place, text or go to shopping options. These then lead into more information, once the device has been identified.

We've seen it:

  • Identify The Shard in London and suggest local restaurants
  • Recognise a box of Lindt Lindor (strawberries and cream flavour) and suggest shopping options
  • Spot a bottle of Casillero di Diablo and offer details on the wine

The best bit about Bixby Vision is that it's not dependent on the full Bixby Voice service, with direct access through the camera, so even those who don't get the full Bixby service will get Bixby Vision.

Pocket-lintwhat is bixby samsung s smart ai explained image 13

Samsung has confirmed that Bixby will be integrated with a number of apps, detailing that it would support 10 native Samsung apps at launch. This is where Bixby will hit rougher ground, because of the variety of apps that you'll have on your phone. Samsung hasn't detailed what those apps will be, but it's confirmed to work with:

  • Phone
  • Messages
  • Internet
  • Gallery
  • Camera
  • Contacts
  • Settings
  • Bixby
  • Reminder
  • Weather

Exactly what Bixby does with Bixby, we don't know.

Samsung has also said that it plans to support third-party apps through an SDK, but the big question is surrounding Google apps. Samsung relies on some core Google apps for the experience it offers - like Play Music and Google Maps, apps that definitely need integration.

Samsung has said that the supported apps will quickly increase, but there's plenty of question here still to be answered.

Pocket-lintwhat is bixby samsung s smart ai explained image 8

There's another side to Bixby. For those not diving into the voice service - or those who can't get it because it's not supported in their country - then there's Bixby Home.

In the UK this integrates the Upday news service (accessed via a left-swipe on the screen or a short press on the button) or Flipboard for those elsewhere and pulls in cards from supported apps. We've seen it serving up weather and S Health steps as well as news highlights.

Samsung also showcased Bixby Home learning from your routine, updating to reflect how you use your phone throughout the day. It also showed an Uber card in Bixby home, suggesting that Uber will be on the list of supported apps. We can imagine "Bixby, call me an Uber" working nicely. 

In essence, Bixby Home looks a lot like Google Now, or the evolved Google Search pages you get that serve up information cards based on your interests and so on.

We've also got Spotify integrated in Bixby Home, ready to play music with a tap, one of the services that is happy to be integrated with Bixby. You can also have CNN, Facebook, Foursquare, Twitter as well as Uber as we mentioned.

Injong Rhee, executive VP and head of R&D software and services at Samsung has said "Bixby will be gradually applied to all our appliances. In the future you would be able to control your air conditioner or TV through Bixby." Rhee goes on to say that because Bixby is a cloud service, all you need is an internet connection and a mic and Bixby could be set to task.

We've also seen some very clever demos with Samsung's 2017 televisions, which also offer full voice control. In a similar demo, we saw a voice command to change the contrast demonstrating the same sort of functionality. There was no mention of Bixby at this demo, but we've asked Samsung if it's part of the same system.

Currently there's no confirmation on other Bixby devices, but we suspect we'll be hearing a lot through the year - IFA 2017 would be an obvious time of Bixbyageddon.

Bixby Pay

Samsung could include technology that will allow you to process payments through voice commands. The feature is supposedly being developed under the name Bixby Pay, according to SamMobile, but there's no confirmation of this yet. 

Samsung has confirmed that Bixby will be launching with support for Korean and US English, although it claims that Bixby is in all devices - it just might not work if you're device is set to a different language. Samsung has also confirmed that US English will launch 1 month after Korean.

Samsung has said that support for British English for the UK and other languages for Europe, but there's no word on when that might happen or what languages will be supported.

At launch the Samsung Galaxy S8 offers Bixby Home, Bixby Vision and Bixby Reminder.

Bixby launches on the Samsung Galaxy S8 and Galaxy S8+.

There's no word on whether Samsung will enable Bixby on Galaxy S7 or the Gear S3 for example, but as Samsung has confirmed that all you need is an internet connection and a microphone, it seems likely that Samsung will enable it for other devices. 

Samsung's ambition is clear to have Bixby on all its devices, so we suspect there will be plenty on the update list.

Bixby was officially confirmed on 20 March via a blog post from Samsung. Bixby will be on the Samsung Galaxy S8, which was announced on 29 March, going on sale around 20 April onwards. The date differs in different territories.

We will keep you updated on all the developments of Bixby as we learn more.