The sub-6 GHz spectrum and the millimeter wave frequency band, with its huge spectrum, have been exploited in the past years to meet the traffic demands of wireless communication networks. However, the limited and licensed spectrum of these bands cannot support the massive connectivity requirements, the exponential growth of data traffic, and the strict quality-of-service requirements for 6G and beyond wireless systems. Extremely high frequencies, such as optical and terahertz, which offer much wider transmission bandwidths with extreme data rate capabilities, are expected to play key roles in the 6G and beyond era. As a result, future-generation wireless networks will transition from single-band and heterogeneous networks to multi-band networks (MBNs), where various frequency bands coexist. Despite the great potential of MBNs, they face novel challenges from channel modeling, transceiver and antenna design, programmable simulation platforms, standardization activities, and resource allocation. This paper provides a tutorial overview from the communication design perspective of the various frequency bands, elaborating on the above issues. Then, we introduce and examine typical MBN architectures for future networks and provide a detailed overview of state-of-the-art resource allocation problems for existing MBNs that typically operate on two frequency bands. The considered resource allocation optimization problems and solution techniques are discussed comprehensively. We then identify key performance metrics and constraint sets that should be considered for resource allocation optimization in future MBNs and provide numerical results to depict how various system parameters and user behaviors can influence their performance. Finally, we present several potential research issues as future work for the design and performance optimization of MBNs.