Skip to the outline
A Rough Idea Instrumented People, Places and Things Modeling Activity: outline

Modeling Activity

Computational Design and the Opportunity for AI Applications

Thomas J. McLeish · Berkeley MDes · November 4, 2026

This is the talk’s working outline, drawn from the author’s position papers. Each line is a slide’s headline, numbered as in the slides, so reading them in order gives the argument. Notes and talking points come next.

Models

  1. Modeling is central to design.
  2. Computational design moves the physical world into the virtual world, and the virtual world into the physical.
  3. Central to computational design is the construction and use of models.
  4. A model is a representation, of selected aspects of the world, constructed from a perspective, and for a purpose.
  5. Every model embodies assumptions about what matters and how the world might work.
  6. The distinction between a model and its use is important.
  7. The following case studies show different ways to make a hypothesis about activity operational in a computational system. Contents

Musicolour

  1. Musicolour embodied a model that encouraged improvisation.
  2. The machine was designed to entrain the performer, and to couple the performer into the system.

Colloquy of Mobiles

  1. Colloquy of Mobiles modeled social interactions through the mechanisms of autonomous agents.
  2. Pask imagined a population of agents whose drives could build and be satisfied through communication and collaboration. Live view
  3. Each mobile had its own state and mechanisms for signaling, responding and adapting. Live view, plan
  4. Interactions with other mobiles could produce the conditions for drive satisfaction and shape subsequent behavior. Live view, Female A
  5. The social pattern was not prescribed as a script; it arose from the mechanisms in the agents and from their exchanges. Live view, from Female A
  6. Colloquy of Mobiles is an agent-based model.
  7. The resulting activity is generated by the model rather than retrieved from observations.
  8. A model specifies a representation and its mechanisms; a simulation is an execution of that model under particular conditions.
  9. A coherent simulation is not, by itself, evidence that the model describes real social behavior.
  10. The 2018 reconstruction embodied those mechanisms in a working physical and computational system. Video
  11. At most, Pask was modeling a self-organizing system with abstract models of human drives and communication. Do we see that when we watch it?

Negroponte

  1. Negroponte explored a model of design activity as a basis for computational conversation.
  2. The machine would need some account of what a designer was doing, not simply a vocabulary of commands and outputs.
  3. The computational system can embody a model of the activity in which it is meant to participate. Diagram

PlaceLab and MITes

  1. PlaceLab and MITes made everyday activities available to systematic observation. Picture
  2. Sensors integrated into the architecture provided evidence about activity. Plans
  3. An observation is not an activity model, but you need one to determine observations. Screenshot
  4. A model relates observations to an account of activity. Columns
  5. Recognition and prediction are possible uses of a model, not proof that its account of the activity is complete.

Iota

  1. Iota developed specific behavioral models through experiments tied to real-world questions.
  2. The work could resemble an experiment in physics. Diagram
  3. Data does not arrive already labeled with the meaning of an activity.
  4. People, places and things provided several ways into the same problem: activity unfolding in relationships among them. Columns
  5. METIS connects concepts to proposed measurable components so that relationships can be investigated. Diagram
  6. The Iota studies ran from 2010 to 2015, anchored on people, places and things. Timeline
  7. Whether a moment was engaging depends on relationships that require interpretation and situated accounts. Picture
  8. These models were built from a participant’s labels against a dataset of sensor events. Picture

Simulations, twins and ghosts

  1. Contemporary vocabulary groups quite different computational representations under the idea of a digital counterpart. Table
  2. These terms suggest different kinds of accountability.
  3. A simulation of the mechanisms of Colloquy is not a live twin of the original installation.
  4. The interesting question is what hypothesis a representation embodies, what evidence supports it, and what it can legitimately be used to do.

The opportunity for AI applications

  1. Computational design relates the world, models, possibilities and changed conditions. Diagram
  2. Language models, multimodal AI, context-aware computing and physical AI expand how applications can interpret evidence and produce responses.
  3. An AI model’s general capability and an application’s model of the particular activity remain different things.
  4. Context means the information and representations of a situation that inform an AI system’s completion, conversation, reasoning or action.
  5. Designing an AI application includes the model of the world it embodies, not only the selection of a powerful foundation model.
  6. The apparent fluency of an AI system is not evidence that it has an adequate model of the situation.
  7. Computational Design may offer an important contribution to AI application development by giving sustained attention to the modeling of activity and behavior.

Conclusion

  1. Modeling activity is one of the ways to make the hypotheses underlying AI applications explicit, examinable and consequential.
  2. As designers, you have models, whether you articulate them or not.
  3. These questions can guide a model of your own: a model as a hypothesis. List

Appendix

  1. The appendix holds the kitchen’s results and the references.
  2. The kitchen’s exploratory evaluation uses one participant’s 54 labeled episodes. Table
  3. This talk rests on these sources. List