Planning and visualization for a smart meeting room assistant 1

Abstract

In this paper, we report on the planning and visualization capabilities of Mr.Jones – a proactive orchestrator and decision-support agent for a collaborative decision making setting embodied by a smart room. The duties of such an agent may range across interactive problem solving with other agents in the environment, developing automated summaries of meetings, visualization of the internal decision-making process, proactive data and resource management, and so on. Specifically, we focus on how the visualization of the planning and plan recognition processes forms a key component of the smart assistant, and establishes transparency in the decision-making process. We also highlight how these processes contribute to the proactive nature of the agent. We demonstrate some of these functionalities in a successful deployment of the system in the CEL – the Cognitive Environments Laboratory at IBM’s T.J. Watson Research Center (Yorktown, USA), and report on emerging deployments of the system that have turned into success stories.

Keywords

Explainable AI visualization smart room planning and decision-making human-in-the-loop

1. Introduction

Meetings are an essential part of almost every productive endeavor today, and research has shown that people spend up to three-quarters of their working lives in meetings; however, meeting productivity numbers rarely breach the 50% mark [19]. Moreover, meetings are not going away anytime soon – indeed, this observation made by Hackman & Kaplan [9] holds true even today: “Almost every time there is a genuinely important decision to be made in an organization, a group is assigned to make it – or at least to counsel and advise the individual who must make it.” In the present era, this motivates the need for AI assistants that will help make meetings more efficient and productive for human agents engaged in a decision-making process.

Previous work on such assistive agents has included both purely software agents [4,25] as well as ones that co-inhabit physical spaces with humans [6] and use their understanding of what is happening in those spaces to act as collaborators on cognitive tasks such as decision making. CELIA – the Cognitive Environments Laboratory Interactive Agent – is an early prototype of such an agent in an embodied environment called the CEL[13]. While CELIA in its present form has been used for hundreds of customer engagements and decision-intensive settings [17], by and large the style of interaction has been command-response; i.e. users address CELIA directly by name and issue very specific instructions and goals. Since CELIA does not currently have the capability to understand the context of an interaction, it is difficult for it to be proactive. To address this gap, we present Mr.Jones which can make human–agent interactions much more immersive and natural by adding proactive detection capabilities to assist decision-making in smart environments.

Some of the primary responsibilities of the agent (Fig. 1) include solving problems interactively with other humans in the environment, generating summaries of events in the workspace, visualizing the decision processes of all the embedded agents and of the group (humans and agents) as a whole, proactively managing resources like background services and data; and so on. We divide the responsibilities of Mr.Jones into two processes – Engage , where plan recognition techniques are used to identify the task in progress; and Orchestrate , which involves active participation in the decision-making process via real-time plan generation, visualization, and monitoring. The Engage process – where Mr.Jones pays attention to many modalities of sensory feedback, including speech and image recognition – is an essential capability of a proactive agent.

An interesting caveat of having a truly proactive agent is the loss of common ground with the human decision-makers – i.e. the humans are no longer in control of the planning and execution process of the agent. This makes it imperative to establish effective channels of communication between the agent and the human(s) interacting with or around it, so that the benefits of collaborative decision-making can be realized. We outline the role of visualization in the operation of Mr.Jones, especially as it relates to the externalization of its cognitive processes – i.e. the “mind” of Mr.Jones. This is required to enhance the transparency of the decision-making process, and to achieve common ground with the humans in the loop. This is especially true in a smart room setting where it is hard to assign events to relevant processes in the agent’s mind – What did it see? What did it hear? What information does it have access to?

In this paper, we report on Mr.Jones as a successfully deployed application of planning technologies, towards realization of a truly proactive decision support agent. Specifically, we show –

how proactive elements of the decision-support agent can be designed in the context of long-term interactions for collaborative decision-making with humans in the loop;

how visualization capabilities of the agent become an integral component towards the support and realization of those capabilities and in establishing common grounds for trust and transparency of its decision-making processes.

We present a suite of demonstrations to illustrate these functionalities from an initial deployment of the system in the CEL – the Cognitive Environments Laboratory at IBM’s T.J. Watson Research Center.

Fig. 1.

The different roles of Mr.Jones as a smart room orchestrator and meeting facilitator.

Fig. 2.

The building blocks of Mr.Jones – the two main components Engage and Orchestrate situates the agent proactively in a decision support setting with human decision makers in the loop.

2. Mr.Jones: An end-to-end planning system

In the following section, we present the design and implementation of Mr.Jones as a proactive decision support agent embedded in a physical space. Unlike previous assistive agents [4], we concentrate only on the facilitation of planning processes in the smart room setup. The decision-making process of Mr.Jones has been divided into two phases – Engage and Orchestrate – keeping in mind their relation to the overall planning process as well. We begin thus with a brief overview of automated planning before we dive into the details of these two phases.

2.1. A brief introduction of planning

Automated Planning is the process of generating courses of action to achieve goals given a description of the world. Informally, planning is usually done with a declarative set of operators or actions in the world that transform states into other states. Goals are a subset of those states. The input to a planner is thus a domain file (describing the environment and operators available to the agent) and a problem file (describing the current state of the world and the goal of the agent). The output or solution is a sequence of operators or a plan that transforms the current state into the goal state. Plan recognition, on the other hand, involves mapping observations (which can be in the form of partial states or plans) to a complete sequence of actions (or a final goal in the case of goal recognition). A detailed treatise of planning can be read in [8]. We posit that automated planning is uniquely positioned to orchestrate the many processes in a smart room environment which require long term sequential decision-making while goal and plan recognition allows the agent to be cognizant of the intentions of the agents in the room and inform its own decision making processes to best suit the needs of the latter. For example, the agent can anticipate that a certain task is going to be performed and start proactively bringing up the required resources for it; or reason about its roles in proceedings based on those anticipated tasks – e.g. it can automatically start recording a meeting if a summary will be requested; or even assist the humans in reminding them missing parts of a process that they might have overlooked (such as in the orchestration of a business process).

For an “end-to-end” planning system, such as the one we describe, the planning problem is built from sensory data and the solution or plan is dispatched to actuators that operate on the real world. Furthermore, for systems with humans in the loop, such as ours, the actuated plan of the agent should consider intentions of agents around it and it follows that the sensory data should also include information that facilitates prediction of the same. In order to achieve this, we separate out the internal decision making processes of Mr.Jones into two parts – first, where it proactively tries to identify and situate itself in the ongoing proceedings in the room using goal recognition and second, where it actively tries to help in those proceedings by use of planning and plan recognition (as illustrated in Fig. 2). This separation makes the entire end-to-end system tractable and allows the agent to run all these inferences in real-time.

2.2. ENGAGE

The Engage process consists of the decision support assistant monitoring various inputs from the world in order to situate itself in the context of the group interaction – that is, the assistant literally engages with the world around it. The process is further divided into two stages which run in a tightly-coupled loop. First, the assistant gathers various inputs like speech transcripts, live images, and the positions of people within the room; these inputs are fed into a higher level symbolic reasoning component constructed from plan recognition algorithms. The second stage involves the assistant acting on the recommendations of the plan recognition step; it (1) requisitions resources and services that may be required to support the most likely tasks based on its recognition; (2) visualizes the decision process – this can depict both the internal process, which is the agent’s own recognition algorithm, and the external process, which is task-dependent; and (3) summarizes the group decision-making process (e.g. a meeting) in various forms. In this way, the Engage stage provides the “proactive” part of the decision support assistant.

2.3. ORCHESTRATE

The Orchestrate process is the decision support assistant’s contribution to the group’s collaborative process. Once the assistant is able to identify and situate itself in the task that is being collaborated upon (during the Engage phase described above), it must contribute to the decision-making. This can be done using standard planning techniques, and can fall under the aegis of one of four actions as shown in Fig. 2. These actions, discussed in more detail in [21], are: (1) execute, where the assistant performs an action or a series of actions related to the task at hand; (2) critique, where the assistant offers recommendations on the actions currently in the collaborative decision sequence; (3) suggest, where the assistant suggests new decisions and actions that can be discussed collaboratively; and (4) explain, where the assistant explains its rationale for adding or suggesting a particular decision. The Orchestrate process thus provides the “support” part of the decision support assistant.

The Engage and Orchestrate processes can be seen as somewhat analogous to the interpretation and steering processes defined in crowdsourcing scenarios in [16,24]. A key difference is that in those works the humans are the final decision makers with the assistant merely supporting the decision making.

Fig. 3.

Flow of control in Mr.Jones – each service runs in parallel and asynchronously to maintain anytime response of all the individual components, supervised by the centralized Task Manager.

2.4. Architecture design

The design of the entire system is shown in Fig. 2, situating Mr.Jones in the overall context of the CEL environment and the operations of CELIA. The central component – the Task Manager – regulates the flow of information and control flow across the modules that manage the various functionalities of the CEL (cf. Fig. 3). These modules are mostly asynchronous in nature and may be: (1) services2

²
Built on top of Watson Conversation and Visual Recognition services on IBM Cloud and other IBM internal services.

processing sensory information from various input devices across different modalities like audio (microphone arrays), video (PTZ cameras/Kinect), motion sensors (Myo/Vive) and so on; (2) services handling the different services of CELIA; and (3) services that attach to the Mr.Jones module. The Task Manager is responsible for keeping track of the current state of the system as well as coordinating actuation based on the information being relayed across these modules. The actuation can be either in the knowledge or belief space, or in the actual physical space itself. The key components of the Mr.Jones module – as illustrated in Fig. 2 – are discussed in more detail next.

2.5. Key components

The design of Mr.Jones follows closely from the delimitation of tasks in the Engage and Orchestrate phases. Aside from the difference in the nature of tasks assigned to the agent in the two phases, this distinction also stems from practical considerations. The system must at once be more responsive to a broader range of tasks, yet remain robust to different levels of noise and uncertainty as well.

2.5.1. Knowledge acquisition/learning

The knowledge contained in the system comes from two sources – (1) the developers and/or users of the service; and (2) the system’s own memory; as illustrated in Fig. 2. One significant barrier towards adoption of higher level reasoning capabilities into such systems has been the lack of familiarity of developers and end users with the inner working of these technologies. With this in mind we provide XML-based modeling interfaces – i.e. a “system config” – where users can easily configure new environments. This configuration contains description of the environment – for example, for a smart room, this includes people who are generally expected to use the room, layout of the room, what different devices are there and where, and so on. That information in turn enables automatic compilation of the files that are internally required by the reasoning engines. Thus system specific information is bootstrapped into the service specifications written by expert developers, and this composite knowledge can be seamlessly transferred across task domains and physical configurations. The memory of the system is contained in the“system logs”, and is intended to provide the rest of the domain knowledge to the planning component. This is meant to enable easily customizable and generalizable configuration of the system with a very small setup phase, as well as to let services continually learn and adapt to the dynamics of a particular environment over time. In this paper, we do not discuss this further, but instead focus more on the decision making processes of the agent.

2.5.2. Goal and plan recognition

Currently, the system employs the probabilistic goal/ plan recognition algorithm from [18] to compute its beliefs over possible tasks that are currently underway in its environment. The algorithm internally casts the plan recognition problem as a planning problem by compiling away observations to the form of actions in a new planning problem. The solution to this new problem enforces the execution of these observation-actions in the observed order. This technique is heavily dependent on the availability of detailed and accurate models. However, it has the advantage of being able to explain the reasoning process being the belief distribution in terms of the possible plans that the agent envisioned using the compilation in [18]. The explanation service (visualized in the next section) is dependent on the underlying algorithm(s) used.

2.5.3. Plan generation

The FAST-DOWNWARD planner [10] provides a suite of solutions to the planning problem. The planner is also required internally by the Recognition Module when using the compilation from [18], or in general to drive some of the orchestration processes (when it needs to recognize or complete a partial plan given a known goal). The planner reuses the compilation from the Recognition Module to compute plans that preserve the current (observed) context (cf. [18]).

Fig. 4.

Snapshot of the mind of Mr.Jones. A video of the system in action can be viewed at – https://youtu.be/ZEHxCKodEGs.

3. Externalizing the mind of Mr.Jones

In the following section, we will concentrate on the visualizations of the planning technologies afforded by Mr.Jones in the CEL – the Cognitive Environments Laboratory – at IBM’s T.J. Watson Research Center. The laboratory acts as a smart environment, equipped with various sensors and actuators to facilitate group decision making and human–agent collaboration. Automated planning techniques, as explained above, form core components in the design of the decision support capabilities of a smart assistant in this setting. However, the ability to plan is not enough if the agent cannot communicate that information effectively to the humans in the loop. Dialog as a means of interfacing with the human decision makers often becomes clumsy in such settings due to difficulty in representing information in natural language or time taken to communicate as such. Instead, we aim to build visual mediums of communication between the planning components and the humans in the loop. Externalizing the various pathways involved in the decision support process is essential to establish trust between the humans and the machine as well as increase situational awareness of the agents. It can allow the humans to be cognizant of the internal state of the assistant and infer rationales behind its decisions. It is also meant to reduce the cognitive burden on the decision makers and improves their situational awareness.

To this end, we externalize the “mind” of Mr.Jones – i.e. the various processes that feed the different capabilities of the agent. A snapshot of the interface is presented in Fig. 4. This snapshot is organized at increasing levels of abstraction –

Raw Inputs – These show the camera feeds and voice capture (speech to text outputs) as received by the system. These help in externalizing what information the system is working with at any point of time and can be used, for example, in debugging at the input level if the system makes a mistake or in determining whether it is receiving enough information to make the right decisions. It is especially useful for an agent like Mr.Jones which is not embodied in a single robot or interface but is part of the environment as a whole, due to which an user can find it difficult to attribute specific events and outcomes to it.

Lower level reasoning – The next layer deals with the first stage of reasoning over these raw inputs – What are the topics being talked about? Who are the agents in the room? Where are they situated? This helps an user identify what knowledge is being extracted from the input layer and fed into the reasoning engines. It is also meant to increase the situational awareness of agents by visually summarizing the contents of the scene at any point of time.

Higher level reasoning – Finally, the topmost layer uses information extracted at the lower level to reason about higher level tasks in the scene. The upper widget thus provides a visualization of the outcome of the plan recognition process, and an explanation of this process in terms of the information extracted in the lower levels (agents in the scene, their positions, speech intents, etc.). This puts in context the agents current understanding of the processes in the scene and reveals the rationale behind its actions.

The interface consists of five widgets. The largest widget on the top shows the various usecases that the CEL is currently set up to support. In our current setup, there are nine such usecases. The widget represents the probability distribution that indicates the confidence of Mr.Jones in the respective task being the one currently being collaborated on, along with a button for the explanations associated with each such belief. The explanations are generated directly from the plans used internally by the recognition module [18] and justify why, given its model of the underlying planning problems, these tasks look likely in terms of plans that achieve those tasks. Model based algorithms are especially useful in providing explanations like this [7,23]. The system is adept at handling uncertainty in its inputs (it is interesting to note that in coming up with an explanatory plan it has announced likely assignments to unknown agents in its space). In the snapshot above, Mr.Jones has placed the maximum confidence in the tour usecase.

Fig. 5.

An example of an orchestration plan, taken with permission from [2]. The system employs a method for visualization as a process of explanation, explained in detail in [2]. it instantiates the “model reconciliation” process [3] with an empty model to determine the minimal subset of domain features that may be required to prove optimality of a plan. This figure illustrates the minimized view of conditions relevant to a plan describing the decision-making process in the smart room. Blue, green and red nodes indicate preconditions, add and delete effects respectively. The conditions from the domain which are not necessary causes for this plan (i.e. the plan is still optimal in a domain without these conditions) are grayed out in the visualization (11 out of a total 30).

Below the largest widget is a set of four widgets, each of which give users a peek into an internal component of Mr.Jones. The first widget, on the top left, presents a wordcloud representation of Mr.Jones’s belief in each of the tasks; the size of the word representing that task corresponds to the probability associated with that task. The second widget, on the top right, shows the agents that are recognized as being in the environment currently – this information is used by the system to determine what kind of task is more likely. This information is obtained from four independent camera feeds that give Mr.Jones an omnispective view of the environment; this information is represented via snapshots (sampled at 10–20 Hz) in the third widget, on the bottom left. In the current example, Mr.Jones has recognized the agents named “Kartik” and “Mishal” in the scenario. Finally, the fourth widget, on the bottom right, provides a wordcloud based summarization of the audio transcript – this provides a succinct representation of things said in the environment in the recent past as captured via the audio channels. This widget is merely a summarization of the full transcript which is fed into the IBM Watson Conversation service to generate observations for the plan recognition module. The interface thus provides a (constantly updating) snapshot of the various sensory and cognitive organs associated with Mr.Jones – the eyes, ears, and mind of CELIA.

4. Emerging deployments: Success stories

Since many of the tasks either involve confidential client data or are not in the public domain yet, we do not provide empirical analysis of the system in any individual domain. In this section, we instead report on four emerging deployments of the Mr.Jones system and various other tools inspired by the work reported in this paper. Each of the deployments detailed here were evaluated as case studies in their own right; and the end results are presented as a bouquet of success stories for the smart assistant that we describe.

4.1. Smart room assistant

The primary evaluation and demonstration of the Mr.Jones system is as an assistant in a smart room setting. In such a setting, Mr.Jones performs the twin roles of engage and orchestrate (defined previously). Below, we show deployments of both these roles. This instantiation of the Mr.Jones system [2] was awarded the Best Demo: Runner-Up prize at the ICAPS 2018 Demonstration Program.

4.1.1. Engage

We demonstrate via a recorded video (link: ibm.biz/jones-demo) how the Engage process evolves as agents interact in the CEL. The demonstration begins with two humans discussing the CEL environment, followed by one agent describing a projection of the Mind of Mr.Jones on the screen. The other agent then discusses how a Mergers and Acquisitions (M&A) task [13] is carried out.

4.1.2. Orchestrate

The orchestrate role of the system is demonstrated by its ability to moderate decision making in multi-entity settings. In such scenarios, the assistant must act as a facilitator of various opinions and preferences among the multiple decision-making entities. Mr.Jones is able to create a plan that starts from determining the participants in the scenario; goes through displaying the various decision alternatives; collecting the participants’ preferences on those alternatives; and handling contingencies like new participants, etc. The smart assistant also displays its orchestration plan to the various participants to ensure shared context. A sample orchestration plan is displayed in Fig. 5.

Fig. 6.

A plan from the exoplanet exploration domain, taken with permission from [12]. The nodes are as follows: dark green – calculated variable; gray – function; and orange – retrieved from database.

4.2. Exoplanet exploration

The next deployment that we describe is the use of a smart assistant to help astrophysicists visualize and analyze data about extra-solar (exo) planets, which are planets outside the traditional Solar System. This assistant – described more fully in a report to appear early next year [12] – enables subject matter experts (in this case, astrophysicists) to interact with data, visualizations, and analyses about exoplanets. The exoplanet assistant provides various capabilities including deixis using speech and gestures; proactive recognition and guidance; interactive query refinement; symbolic model discovery; and explainable self-programming using automated planning techniques.

For example, given a user query about computing a specific quantity – e.g. the stellar luminosity – the assistant can return not just the computed quantity, but the provenance of the calculation as an explanation. This explanation, visualized in Fig. 6, consists of the steps taken by the planner to compute the desired quantity. A full demonstration of the smart assistant in action can be viewed via the following URL: ibm.biz/tyson-demo. A preliminary, deployed version of this instantiation of the smart assistant was awarded Best Systems Demonstration [11] at AAAI 2018.

4.3. Automated meeting summarization

The Mr.Jones smart assistant can be used to produce a multi-modal “summary” of meetings and other activities that it is able to record and transcribe. Specifically, the summarization process is a representation of the beliefs of the agent with regard to what is going on in its space over the course of a given activity. Since the agent already needs to keep track of this information in order to make its engagement and orchestration decisions effectively, an externalization of such information can also be replayed or sampled from to generate an automated visual summary of (the agent’s belief of) the proceedings. After the interaction is complete Mr.Jones thus automatically compiles a summarization (or minutes) of the meeting by sampling from the visualization of its beliefs. A video illustrating a typical summary can be accessed at ibm.biz/jones-summary.

This kind of visual summary provides a powerful alternative to established meeting summarization tools like text-based minutes. The visual summary can also be used to extract abstract insights about this one meeting, or a set of similar meetings together and allows for agents that may have missed the meeting to catch up on the proceedings. Whilst merely sampling the visualization at discrete time-intervals serves as a powerful tool towards automated summary generation, we anticipate the use of more sophisticated visualization [5] and summarization [14,15,22] techniques in the future.

5. Conclusion

In this paper, we presented Mr.Jones, a planning-based agent that assists with decision-support tasks in smart environments. We introduced the Engage and Orchestrate processes that need to be supported by any planning agent that provides proactive decision-support in team settings; and then detailed the implementation details of Mr.Jones that support these two processes. We demonstrated the integration of goal and plan recognition capabilities that are used by the system to situate itself proactively in a collaborative task scenario, and to produce a summary of that interaction. We also demonstrated recent success stories that involve initial deployments of the system in various AI use-cases. Going forward, from the point of view of collaborative planning interfaces, we hope to take inspiration from extensions of decision support interfaces in [21] to the newly emerging mixed-reality paradigm in [20] and explore newer avenues for human-planner interaction. We also plan to integrate our system with more existing planning tools and make it available as a service. We hope that this will lead to a future where the long-standing vision of a one-stop, online, domain-independent tool for automated planning – from model creation and acquisition, to plan visualization and explanation – is realized.

Footnotes

Acknowledgements

We would like to gratefully acknowledge Jason Ellis, Victor Dibia, and Andy Aaron for their help with this work and the various deployments in the CEL.

References

Chakraborti,

K.P.

Fadnis,

Talamadupula,

Dholakia,

Srivastava,

J.O.

Kephart and

R.K.E.

Bellamy, Visualizations for an explainable planning agent, in: Proceedings of the Twenty-Seventh International Joint Conference on Artificial Intelligence, IJCAI-18, International Joint Conferences on Artificial Intelligence Organization, 2018, pp. 5820–5822. doi:10.24963/ijcai.2018/849.

Chakraborti,

K.P.

Fadnis,

Talamadupula,

Dholakia,

Srivastava,

J.O.

Kephart and

R.K.E.

Bellamy, Visualizations for an explainable planning agent, in: ICAPS 2018 Systems Demonstrations Program, 2018.

Chakraborti,

Sreedharan,

Zhang and

Kambhampati, Plan explanations as model reconciliation: Moving beyond explanation as soliloquy, in: IJCAI, 2017.

Chen,

Houston,

Nunamaker and

Yen, Toward intelligent meeting agents, Computer29(8) (1996), 62–70, ISSN 0018-9162. doi:10.1109/2.532047.

Dörk,

Gruen,

Williamson and

Carpendale, A visual backchannel for large-scale events, IEEE Transactions on Visualization and Computer Graphics (2010).

Farrell,

Lenchner,

Kephart,

Webb,

Muller,

Erickson,

Melville,

Bellamy,

Gruen,

Connellet al., Symbiotic cognitive computing, AI Mag (2016).

Fox,

Long and

Magazzeni, Explainable planning, in: IJCAI XAI Workshop, 2017.

Ghallab,

Nau and

Traverso, Automated Planning: Theory and Practice, Elsevier, 2004.

J.R.

Hackman and

R.E.

Kaplan, Interventions into group process: An approach to improving the effectiveness of groups, Decision Sciences5(3) (1974), 459–480. doi:10.1111/j.1540-5915.1974.tb00631.x.

10.

Helmert, The fast downward planning system, Journal of Artificial Intelligence Research26 (2006), 191–246. doi:10.1613/jair.1705.

11.

J.O.

Kephart,

V.C.

Dibia,

Ellis,

Srivastava,

Talamadupula and

Dholakia, A cognitive assistant for visualizing and analyzing exoplanets, in: AAAI 2018 Systems Demonstrations Program, 2018.

12.

J.O.

Kephart,

V.C.

Dibia,

Ellis,

Srivastava,

Talamadupula and

Dholakia, An embodied cognitive assistant for visualizing and analyzing exoplanet data, IEEE Internet Computing Journal (2019).

13.

J.O.

Kephart and

Lenchner, A symbiotic cognitive computing perspective on autonomic computing, in: ICAC, 2015.

14.

Kim,

C.M.

Chacha and

J.A.

Shah, Inferring team task plans from human meetings: A generative modeling approach with logic-based prior (2015).

15.

Kim and

J.A.

Shah, Improving team’s consistency of understanding in meetings, IEEE Transactions on Human–Machine Systems46(5) (2016), 625–637. doi:10.1109/THMS.2016.2547186.

16.

Manikonda,

Chakraborti,

Talamadupula and

Kambhampati, Herding the crowd: Using automated planning for better crowdsourced planning, Journal of Human Computation (2017).

17.

Murphy, Searching for Eureka: IBM’s path back to greatness, and how it could change the world, Quartz, 2015.

18.

Ramirez and

Geffner, Probabilistic plan recognition using off-the-shelf classical planners, in: AAAI, 2010.

19.

N.C.

Romano and

J.F.

Nunamaker, Meeting analysis: Findings from research and practice, in: Proceedings of the 34th Annual Hawaii International Conference on System Sciences, 2001, IEEE, 2001, p. 13.

20.

Sengupta,

Chakraborti and

Kambhampati, MA-RADAR – A mixed-reality interface for collaborative decision making, in: ICAPS UISP, 2018.

21.

Sengupta,

Chakraborti,

Sreedharan and

Kambhampati, RADAR – A proactive decision support system for human-in-the-loop planning, in: AAAI Fall Symposium on Human–Agent Groups, 2017.

22.

Shaw, How Wimbledon is using IBM Watson AI to power highlights, analytics and enriched fan experiences, IBM, 2017.

23.

Sohrabi,

J.A.

Baier and

S.A.

McIlraith, Preferred explanations: Theory and generation via planning, in: AAAI, 2011.

24.

Talamadupula,

Kambhampati,

Hu,

Nguyen and

H.H.

Zhuo, Herding the crowd: Automated planning for crowdsourced planning, in: HCOMP, 2013.

25.

L.L.

Voss and

Ehlen, The CALO meeting assistant, in: Proceedings of Human Language Technologies: The Annual Conference of the North American Chapter of the Association for Computational Linguistics: Demonstrations, Association for Computational Linguistics, 2007, pp. 17–18.

Planning and visualization for a smart meeting room assistant 1

Abstract

Keywords

1. Introduction

2.1. A brief introduction of planning

2.2. ENGAGE

2.3. ORCHESTRATE

2 Built on top of Watson Conversation and Visual Recognition services on IBM Cloud and other IBM internal services.

2.5.1. Knowledge acquisition/learning

2.5.2. Goal and plan recognition

2.5.3. Plan generation

4.1. Smart room assistant

4.1.1. Engage

4.1.2. Orchestrate

4.3. Automated meeting summarization

5. Conclusion

Footnotes

Acknowledgements

References

²
Built on top of Watson Conversation and Visual Recognition services on IBM Cloud and other IBM internal services.