dojself04

[태그:] TPE

  • * Tree-Structured Parzen Estimator: Understanding Its Algorithm Components and Their Roles for Better Empirical Performance (ArXiv 2023)

    * Tree-Structured Parzen Estimator: Understanding Its Algorithm Components and Their Roles for Better Empirical Performance (ArXiv 2023)

    1. 문제 설정: Black-box Optimization TPE는 Bayesian Optimization (BO) 계열 알고리즘으로, 다음 문제를 해결합니다: x∗=arg⁡minx∈𝒳⁡f(x)x^* = \arg\min_{x \in \mathcal{X}} f(x) 2. 핵심 아이디어: 기존 BO와의 차이 일반 BO (e.g., GP-based) TPE의 핵심 차별점 역방향 모델링: p(x|y) 를 직접 모델링p(x|y) \text{ 를 직접 모델링} 즉, p(x|y)={p(x|𝒟(l))(y≤yγ)p(x|𝒟(g))(y>yγ)p(x|y) = \begin{cases} p(x | \mathcal{D}^{(l)}) & (y \le y_\gamma) \\ p(x | \mathcal{D}^{(g)})…

  • * Optimizing Instructions and Demonstrations for Multi-Stage Language Model Programs (EMNLP 2024)

    * Optimizing Instructions and Demonstrations for Multi-Stage Language Model Programs (EMNLP 2024)

    핵심 질문 여러 단계로 구성된 LM pipeline에서 instruction + few-shot demo를 어떻게 jointly 최적화할 것인가? 1. 문제 설정 (Problem Formulation) LM Program 정의 목표 전체 프로그램 성능을 최대화: Φ∗=arg⁡maxV→S⁡𝔼(x,x′)∼Dμ(ΦV→S(x),x′)\Phi^* = \arg\max_{V \to S} \mathbb{E}_{(x,x’) \sim D} \mu(\Phi_{V \to S}(x), x’) 중요한 점: 즉, credit assignment problem + combinatorial search 2. 핵심 문제 (Challenges) 논문에서 명확히…