The origins of the Library can be traced back to 1926 with the establishment of the Junior College of the Lower Rio Grande Valley which later was renamed as Brownsville Junior College in 1931. The library of these early Brownsville institutions of higher learning was located in the Brownsville High School on Palm Boulevard, and was part of the high school library. In 1948 the Library was relocated to a wing of Gorgas Hall on what was previously the site of Fort Brown. Fort Brown was de-activated in 1947 and purchased by the City of Brownsville and the Brownsville Independent School District resulting in the relocation of Brownsville Junior College from Brownsville High School to the grounds of old Fort Brown. By 1954 the Library, now named the Zachary Taylor Library, was moved into Stockely Hall within the Jacob Brown Civic Center. The Library served the needs of both Texas Southmost College (the new institutional name for Brownsville Junior College as of 1949) and the City of Brownsville. During the 1960's an additional building served as a “reserve” library. This building was a surviving Fort Brown building (no longer standing) which was located in Front of Gorgas Hall and briefly served to supplement the library facility at the Jacob Brown Civic Center by providing students with materials placed on reserve by college instructors and as a storage area for the library's journal collection. In 1966 the Library again relocated, moving from Stockely Hall, to the beginnings of the present day structure. In order to facilitate the growth of the library’s book collection, a second floor was added in 1975. The Library was renamed as the Arnulfo L. Oliveira Memorial Library in 1980 upon the untimely death of Dr. Oliveira, formerly a president of Texas Southmost College. By 1983 the Library collection was converted from the Dewey Decimal Classification System to the Library of Congress Classification System. During the late 1980’s the Library retired its card catalog and implemented its first online catalog. The partnership between Texas Southmost College and the University of Texas at Brownsville was enacted by the Texas Legislature in 1991 and the Library then became a part of the University of Texas System. During the period of 1994-95, the City of Brownsville withdrew its material from the Library in order to relocate its collection to the present day Brownsville Public Library and the Library at this point, became solely an academic library. During the evolution of the Library from its origins with the Junior College of the Lower Rio Grande Valley in 1926 to its current role serving the University of Texas at Brownsville in partnership with Texas Southmost College, the Library's collection has expanded from that of a predominately print collection to that of a major resource center providing online databases, computer files and audiovisual materials in addition to its print collection.

The mission of the University Libraries at The University of Texas at Brownsville and Texas Southmost College is to combine the strengths of a community college library with those of an upper-level and graduate library to enhance and facilitate the dissemination of information—through print, media, audio and electronic resources—to faculty, staff and students, thereby meeting their educational needs. The library’s goal is to provide quality services to respond to these needs. The library’s objective is to provide electronic bibliographic access to the best materials within its own system and to materials located elsewhere.  The University of Texas at Brownsville is part of the University of Texas System and has access to a myriad of resources beyond its own region.  Because the University offers graduate work, the intention of the library is to provide resources that are substantially beyond those required for its baccalaureate programs and to make every effort to develop further holdings that will support new degree programs. The goal of the University Libraries is to play an essential role in the teaching and learning mission of the University of Texas at Brownsville and Texas Southmost College. The library consistently strives to update its computer technology and develop efficiently its print and non-print collection to meet the scholastic needs of its users.