The club was born from humble beginnings as an Army Hut in Chalmers St (in the current position), on the Swansea Channel for the Swansea RSL Sub-Branch around 1936.
The club was for Sub-Branch members only and then later opened to social members with the Committee and staff all voluntary workers. The first official Club Board was elected in 1947 with the first Secretary Manager appointed in 1974. The clubs development commenced around 1961 with a more modern brick building on one level situated in Chalmers St. This then followed in 1987 with a two story building with the famous “Hole in the Floor” feature with gaming upstairs and the main auditorium.
There have been two further major developments, in 1997 saw the entry of the club changed to Bridge Street opening lounges areas and eateries to the water views of the Swansea Channel. With a land swap between the Memorial Hall Trust and the RSL Club saw the old 1932 Memorial Hall building demolished and the new Hall built in Bridge St for the general use of the community.
In 2002 the club purchased the land from the Memorial Park Trust, Built a new Memorial for the returned service personnel. Then in 2007 the club extensions continued with a new Administration, Tab and Outdoor Smoking areas as well as new refurbishment of the Lounge, Bar and Poker Machine areas.
In 2006 the name of the club was changed from Swansea RSL Sub-Branch Club Ltd to Swansea RSL Club Ltd.
The club has 8020 members and employs 65 staff and is a large contributor to the community of Swansea and the surrounding communities in many ways. The club has sponsored the famous New Years Eve Fireworks at Blackneds Bay from 2000 until 2010, it also supports all the Surf Life Saving Clubs and many of the local sporting clubs.