Reinforcement learning from formal specifications often separates task definition, reward shaping, curriculum design, and policy diagnostics into different mathematical objects. This fragmentation introduces additional conversion steps and discards structure already present in the specification, such as sub-task boundaries, ordering constraints, and quantitative margins. We introduce TBTCL, a framework that uses Temporal Behavior Trees (TBTs) as a unified interface for specification-guided reinforcement learning. A single TBT specifies the task, induces dense robustness-based rewards, defines a curriculum, and supports better understanding of learned policies - grounded in the formal semantics of the specification language. By adapting TBT syntax and semantics for specification-guided reinforcement learning, we provide a formalism that is more expressive than popular specification languages such as LTL and inherently hierarchical. We present algorithms to automatically extract a curriculum DAG and derive dense, robustness-based rewards directly from TBT semantics, without requiring additional user inputs such as region predicates or demonstrations. Furthermore, we utilize TBT trace segmentation to build understandable competence profiles and automatically adapt the reward toward bottlenecks. We evaluate TBTCL on six specifications across discrete and continuous domains. The results show that TBTCL matches dense STL shaping on simpler tasks and improves over baselines on reach-avoid specifications, while also exposing interpretable sub-task failures through the same formal object used for training. We show that a Temporal Behavior Tree can serve as a specification, a reward, a curriculum, and a diagnostic.