The Church of All Saints is one of the valuable samples of crazy Baroque churches belonging to the Roman Catholic concessions. Included in the ensemble novitiate (for novices) and a Carmelite monastery in the old city.

The Church was built together with the monastery for 11 years from 1620 to 1631 year near Rudnicki gate. During the hostilities with Moscow, the temple burned and renovation in 1655 was substantially rebuilt. Later, in 1743, near the North-Eastern corner, the site of the towers, was built of complex shape bell tower in Tiel late Baroque. In 1812 the Church was seriously undermined by Napoleon's soldiers, who burned confessionals and pews. The French used in the temple hospital. The renovated Church was in 1823.

The Russian authorities abolished the monastery, and in 1885 in the premises of the monastery staged a cute apartment, and since 1948 has closed and the Church, having arranged in it a stock of groceries.

From 1967 to 1975 in the temple restoration work was carried out under the guidance of architect Aldona Suboscine. After the restoration of the temple acted as the Museum of Lithuanian folk art. Restitution of the Church was held immediately after the change of regime in 1990 the Church was returned to believers and operates to this day.

A building plan of the Church has the shape of a Latin cross, the shape of the Church is three-nave Basilica type. The originality of the space inside the temple that the side aisles formed by the lateral chapels of the Church. The side aisles 3 times already and 2 times below the centre, separated from it by two pairs of pylons on each side. The arches of the nave are cylindrical with lunettes.

The main facade has a form architecture of the early Baroque style, the facade is divided into two tiers cornice, pilasters distinguish it from the vertical axis. Crowns the facade pediment a triangular shape, stands on the sides of the obelisks. The portal in the Renaissance style emphasizes the Central axis of the building. In the niches previously installed the statues of the founders of the Carmelite monasticism – St. Elias, and St. Elisha, made of wood.

A monumental four-storey bell tower, extending downward, ends with the helmet and openwork cross. The pilasters of the lower tier, decorated with rust, making a vivid contrast with other fixtures in the corners of the columns. Corinthian pilasters of the second tier is decorated with stucco motif. In the third tier there are obliquely lateral pilasters flank the columns. At last, the fourth tier of the pilasters as it grows out of the helix.

The window niches of the bell tower have different arch shape and decorated with stucco work, and in the fourth tier of the niche is still enclosed within decorative balcony railings. It is assumed that the bell tower built by the same architect that built the tower of the Carmelite Church in Belarus.

The walls and vaults of the naves of the temple, the dome of the side chapels are decorated with frescoes and ornaments, the frescoes depict scenes from the lives of saints and the history of Lithuania. Decorative moldings that adorn the interior of the Church painted in the late 18th century. In 18 Church altars, decorated with sculptures of saints and frescoes depicting the stories of their lives. The main altar was built probably in the late 1780s, project Martin Confusa.

During the restoration works carried out in 1902 on the initiative of the priest Chudovsky, murals were painted over, today opened only a small portion of them.