Wood quality is predominantly determined by the amount and the composition of secondary cell walls (SCWs). Consequently, unraveling the molecular regulatory mechanisms governing SCW formation is of paramount importance for genetic engineering aimed at enhancing wood properties. Although SCW formation is known to be governed by a hierarchical gene regulatory network (HGRN), our understanding of how a HGRN operates and regulates the formation of heterogeneous SCWs for plant development and adaption to ever-changing environment remains limited. In this review, we examined the HGRNs governing SCW formation and highlighted the significant key differences between herbaceous Arabidopsis and woody plant poplar. We clarified many confusions in existing literatures regarding the HGRNs and their orthologous gene names and functions. Additionally, we revealed many network motifs including feed-forward loops, feed-back loops, and negative and positive autoregulation in the HGRNs. We also conducted a thorough review of post-transcriptional and post-translational aspects, protein–protein interactions, and epigenetic modifications of the HGRNs. Furthermore, we summarized how the HGRNs respond to environmental factors and cues, influencing SCW biosynthesis through regulatory cascades, including many regulatory chains, wiring regulations, and network motifs. Finally, we highlighted the future research directions for gaining a further understanding of molecular regulatory mechanisms underlying SCW formation.