The software secretaries

By A.S.

A PEPPY crowd poured into the Bill Graham Civic Auditorium in San Francisco on September 9th. They had turned up at the venue, which usually hosts rock concerts, to watch a show by Apple, a firm with a cult following of its own. Executives took the stage to unveil updates to the Apple Watch, iPhone and iPad, as well as a new television set-top box, which makes it easier to find TV shows and play games.

The real star of the show, however, was Siri, Apple’s personal-assistant technology. Siri, which is already available on iPhones, responds to voice commands and will be embedded in Apple’s new TV remote controls, so users will not have to lift a finger to change channel or find new shows. If users want to know the weather or the results of a sporting event they can ask Siri, and be met with a rapid reply.

Siri’s migration to television exemplifies the rise of virtual-assistant software that mimics some of the skills of human secretaries: creating reminders for appointments, looking up information and completing other tasks. Apple, which bought Siri in 2010 for an estimated $200m, has been a leader in this area, but many tech firms have been working on rival products.

Google and Microsoft offer digital assistants on smartphones, called Google Now and Cortana, respectively, which have deep knowledge of their users’ habits and schedules. Amazon sells a stand-alone device that, among other things, plays music, reads books aloud and can help buy items through Amazon. On September 8th Baidu, a Chinese internet giant, announced its own digital agent, Duer. And recently Facebook announced that a concierge service, called M, would be available through its messaging app. To start with, tasks will be completed by humans, but that could change over time.

All these firms still have kinks to work out, but the sophistication and range of functions of their personal-assistant software are expanding (see table). The rise of Siri et al signals two important trends that will shape the future of the consumer internet: the evolution of “search”, away from typed search-engine queries towards a more personalised, interactive service, and a gradual shift from individual apps to an ecosystem of services that is mediated by a powerful software assistant.

According to Gartner, a research firm, around 38% of American consumers have used virtual-assistant services on their smartphones recently; by the end of 2016 an estimated two-thirds of consumers in developed markets will use them daily. Software robots are getting better at predicting what users need based on past behaviour and current location.

The virtual secretaries offered by a variety of tech giants form part of their wider efforts to master artificial intelligence and, in particular, “machine learning”—teaching computers to crunch vast amounts of data, recognise patterns and get better at what they do. Firms are spending billions of dollars buying startups in this area, and are rushing to hire specialists in artificial intelligence. Apple alone is reportedly looking to hire around 90 experts, and Facebook has hired a star artificial-intelligence academic, Yann LeCun, to head its research centre on the subject.

Voice recognition is quickly improving, although it remains imperfect. Two years ago Google Now used to misinterpret around 25% of words spoken, but today it only misses 8%, according to Aparna Chennapragada, who oversees the product. Firms’ big focus has become how to use the information consumers store on their devices to make proactive recommendations instead of just responding to requests.

Google is especially good/creepy at this. For example, it goes through users’ e-mails to prompt them about when to leave for appointments or flights. Microsoft’s Cortana does this too but until recently it has been limited to Windows devices. Global-positioning technology, already a standard feature on smartphones, is helping virtual assistants to do their job better. If a consumer wants to be reminded to buy milk when he next goes to the supermarket, an alert will flash when the phone detects that he has arrived.

Technology is also making it possible to automate the headache of arranging meetings. A few startups, including Clara Labs and x.ai, offer virtual scheduling assistants that employ a combination of algorithms and humans to help set up appointments. Subscribers copy in the software robot on e-mails; it scans their calendars and chooses a convenient slot on their behalf. On average it takes humans around seven e-mails to set up a meeting, so virtual assistants can save lots of time. People tend to be nice to virtual assistants, even when they know they are corresponding with a robot, says Dennis Mortensen, the boss of x.ai. In time, the software will become so good they may never know their interlocutor is a machine.

Most real-life personal assistants do not need to worry about their jobs being threatened by technology—at least yet. Digital assistants serve as a hands-free way to search for information but struggle to complete actions that require more complex steps, such as booking flights. Your correspondent spent several days “employing” a variety of available digital assistants. Although they can find nearby restaurants and make a booking through OpenTable, they do not yet know her well enough or have the judgment to tell her which is the “best” Italian food. Echo, Amazon’s product, can recite the definition of “personal assistant” without hesitation, but had no idea how to respond when your correspondent asked whether Echo was her personal assistant.

To please their masters, virtual assistants will need to do better at connecting to outside services to fulfil requests. This is the vision of Viv, a startup led by the founders of Siri, which is expected to launch a service in this vein next year. Existing virtual assistants are also making steps to open up. Cortana links to Uber, for example, to help its users book cars, and recently started to show them relevant coupons when they go to shops or browse online.

The rise of virtual assistants poses several challenges. One is privacy. The best services will have access to troves of data, but how consumers’ information will be shared with outside firms that fulfil requests has yet to be ironed out. Unlike human secretaries who work for one person or firm, virtual assistants have divided loyalties. For example, Google Now and Facebook M may work on behalf of consumers, but their parent companies make money by selling targeted ads and mining consumer data. Apple, in contrast, has made privacy one of its main selling-points.

A similar dilemma is raised when it comes to commerce. As virtual assistants help purchase more things online, they will have the power to boost certain firms and deprive others of business. When asked to book a ticket from San Francisco to London, will a virtual assistant select the cheapest fare, or a ticket on an airline that has a promotional relationship with the company that invented the assistant? As virtual secretaries become more powerful, the question of who works for whom will become more pressing.