Taejong Joo

  We develop Graph of Agents, a multi-agent system that expands the context window of large language models by orders of magnitude without any additional training. By framing long context modeling as a compression problem, GoA dynamically builds an optimal multi-agent collaboration structure tailored to the input. This principled approach eliminates the need for complex prompt engineering and specialized multi-agent system designs.

  We demonstrate that the convenience of in-context learning, which enables adaptation to a new task by conditioning on demonstrations in an input prompt, comes with a hidden cost. While it matches the efficiency of a Bayes-optimal estimator in few-shot settings, we prove its statistical efficiency fundamentally diminishes over long contexts. This reveals a critical trade-off between ICL's flexibility and its long-term statistical efficiency.

  We prove that selectively promoting temporal consistency for confident predictions significantly enhances self-training performance under distribution shifts. This approach prevents the common issue of model collapse—where performance deteriorates after a few epochs of self-training—resulting in improved performances with attractive robustness properties.

  We introduce a new approach for simultaneously addressing model calibration and model selection in unsupervised domain adaptation: estimating the average accuracy across subpopulations. For efficient and accurate subpopulation accuracy estimation, we formulate the high-dimensional importance weight estimation problem into a more tractable coordinate-wise convex optimization problem.

  We propose a scalable variational inference framework using a last-layer Dirichlet model as a new alternative to Bayesian neural networks. Our approach significantly enhances uncertainty representation ability of deterministic neural networks while preserving their strong generalization performances and efficiency unlike Monte Carlo dropout and deep ensembles.