The Athenæum's slender, four-story Federal-style building at 9 Market Square, designed by New Hampshire native builder-architect Bradbury Johnson, was completed in 1805 for the New Hampshire Fire and Marine Insurance Company. It is the centerpiece of a row of brick commercial buildings framing the historic city square and its façade is distinguished by tall, arched first-story door and window openings, four attenuated pilasters with composite capitals, and sweeping cornices. The 1805 Building was listed on the National Register of Historic Places on May 24, 1973.
The library first occupied rented rooms and free garret space in Portsmouth until 1823. In that year, faced with the impending dissolution of the insurance company, a group of subscribers to its newspaper reading room purchased the insurance company building and transferred it to the Athenæum in return for newly issued shares. The first floor continued to be a reading room, while the Athenæum's books and museum objects moved into the building's upper floors, which had originally housed St. John's No.1 Masonic Lodge.
The first floor of the building has been a reading room since it was built for the insurance company in 1805. In an 1879 memoir, Proprietor Frank W. Hackett wrote: "It was the custom of the prominent business and professional men of the town, with a sprinkling of retired sea captains, at the close of the business day to drop into the Athenæum for a social chat. Here minds of diverse training and habits of thought got the benefit of mutual attrition [sic], the scholar coming down from the library to compare his views with that of the man of affairs; and conversation took on the character of an interchange of ideas, never degenerating into mere idle gossip."
In the early years the Athenæum also maintained a museum featuring collections typical of that age - mineral and botanical specimens, curiosities from around the world, coins, portraits, and ship paintings and models. While the natural history collections have long since been discontinued, the Athenæum has expanded its collections of fine art and artifacts related to maritime and regional history.
The library now serves a large and diverse audience. In recent years, a new commitment to expand the Athenæum's role as a community resource has greatly expanded public access to its collections, exhibits, lectures, and other cultural programs. The Athenæum also serves as a meeting place for those who wish to exchange ideas and information about diverse aspects of history and culture.