We enable autonomous robot teams to learn both how to act and how to collaborate—deciding which agents should coordinate, with whom, when, and under what resource, uncertainty, and reliability constraints. To this end, we pioneer Value-of-Coordination metrics and mission-grounded distributed optimization methods for cooperative missions of combinatorial complexity, with applications to intelligence, surveillance, target acquisition, and reconnaissance (ISTAR).