In Tilraukot you can find the ruins of the ancient city Kapilvastu which was the capital of the Sakya kingdom that Gautama Buddha was the prince of. If Tilraukot is really the ancient city of Kapilvastu is controversial. Some believe Piprahawa in India is the ancient Kapilvastu.

There is a museum that shows coins, jewelry and pottery from between 700 BC to 400 AD.

Sources: 

Locating the lost kingdom of Kapilvastu - The Himalayan Voice