Stanford Open Virtual Assistant Lab

OVAL Mission

Stanford Open Virtual Assistant Lab (OVAL) is dedicated to advancing virtual assistant technology and creating an industrial ecosystem that protects consumer privacy and promotes open competition.

Fundamental research in virtual assistant technology
Accelerate development through open collaboration
Increase adoption by making quality software openly available
Enable sharing with privacy

OVAL is supported by the National Science Foundation under Grant No. 1900638, and by the Alfred P. Sloan Foundation.

Research Goals

To create an open, non-proprietary linguistic web.
To accelerate and democratize natural language interface technology via open collaboration.
To protect privacy with interoperable virtual assistants that can run on users' devices.

Research

Natural language programming

A new virtual assistant architecture where a neural semantic parser directly translates natural language into a custom high-level programming language (ThingTalk) for assistants. Users can use natural language to automate workflows by composing functions from different domains.

Thingpedia

A WORA (Write-Once-and-Run-Anywhere) platform of skills that can be run on any virtual assistants; these skill can be automatically made available to Alexa and Google Assistant if desired. Thingpedia skills are interoperable to support natural language programming. By minimizing redundant effort, lowering the barrier of entry for new assistants, and hence removing dependency on a small number of proprietary assistants, Thingpedia can flourish to contain as many skills as there are web pages.

LUInet

An open-source neural semantic parser that translates natural language into formal languages. All the training data are also publicly available. Companies can incorporate LUInet in their own virtual assistants, websites, apps, and and voice services.

Genie

A tool that lets non-ML-experts extend Thingpedia and Thingtalk, and craft training data to extend LUInet with sentences in their domains. By empowering companies to create and own interfaces for their domains, LUInet can become more knowledgeable than any proprietary models. This democratizes AI and accelerates AI development.

Almond

The first open-source, social virtual assistant that can be run on local devices to protect privacy. Users can tell their assistant who, what, when, where, and how can their data can be shared, all without disclosing their data to a third party.

A virtual assistant communication protocol

A general, secure protocol based on remote ThingTalk programs to enable interoperability of virtual assistants. This supports federation of assistants, similar to email, rather than centralized monopolies.

Brassau

Brassau combines voice with automatically generated graphical user interfaces to take advantage of the best of each modality.

Blockchain-based technology for audits of third-party sharing

We use communicating virtual assistants to enforce user-specified sharing contracts across institutes; these contracts are made revocable and auditable through efficient blockchain technology based on federated Byzantine agreements.

Presentations & Interviews

"Stanford Team Aims at Alexa and Siri With a Privacy-Minded Alternative", John Markoff for the New York Times, 6/14/2019
"Privacy in the age of virtual assistants", Russ Altman for the Future of Everything (podcast), 6/4/2019
"Giving Control Back to Consumers with Open Federated Virtual Assistants", Monica Lam, Keynote at HiPEAC 2019, 1/21/2019
"Keeping the Internet Open with an Open-Source Virtual Assistant", Monica Lam, Keynote at Mobicom 2018, 11/1/2018

Publications

2020

Zero-Shot Transfer Learning with Synthesized Data for Multi-Domain Dialogue State Tracking
Giovanni Campagna, Agata Foryciarz, Mehrad Moradshahi, and Monica S. Lam
In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), July 2020.
Soundr: Head Position and Orientation Prediction Using a Microphone Array
Jackie Yang, Gaurab Banerjee, Vishesh Gupta, Monica S. Lam, and James A. Landay
In CHI '20: Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems, Honolulu, HI, April 2020.

2019

Genie: A Generator of Natural Language Semantic Parsers for Virtual Assistant Commands
Giovanni Campagna, Silei Xu, Mehrad Moradshahi, Richard Socher, and Monica S. Lam
In Proceedings of the 40th ACM SIGPLAN Conference on Programming Language Design and Implementation, Phoenix, AZ, June 2019.
Artifact — Dataset

2018

Controlling Fine-Grain Sharing in Natural Language with a Virtual Assistant
Giovanni Campagna, Silei Xu, Rakesh Ramesh, Michael Fischer, and Monica S. Lam
In Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies (IMWUT), 2018.
Brassau: Automatically Generating Graphical User Interfaces for Virtual Assistants
Michael Fischer, Giovanni Campagna, Silei Xu, and Monica S. Lam
In 20th International Conference on Human-Computer Interaction with Mobile Devices and Services. (MobileHCI), 2018.

2017

Almond: The Architecture of an Open, Crowdsourced, Privacy-Preserving, Programmable Virtual Assistant
Giovanni Campagna, Rakesh Ramesh, Silei Xu, Michael Fischer, and Monica S. Lam
In Proceedings of the 26th International World Wide Web Conference (WWW), Perth, Australia, April 2017.