This paper proposes a starting point for understanding the material of information architecture practice, by answering the question, "What are we architecting when we practice information architecture?" I propose in summary form some ideas about how information architecture's medium, information, can be usefully described in three modalities (physical, semantic, and digital), and how a full understanding of embodied cognition and affordance theory can help us connect the abstraction of language with structuring concrete, bodily experience.