1st January which marks the beginning of the New Year is also celebrated by Hindus. Many of them visit temples on the eve of New Year. However, astrologically this day has no significance. Visiting temple on New Year day is as good as visiting it on any other day.
In most traditional or religious calendars, the first day of the year is decided based on some celestial event. These calendars use new moon day, full moon day or the transition of the Sun from one zodiac to another to mark the first day of the year. Few traditional calendars also consider Equinox or Solstice to mark the first day of the year. Gregorian calendar which is solely based on human convenience has no such strings attached.
In India the first day of the year, in various regional calendars, is significant because it is based on some celestial event. The first day of the year based on lunar calendars, including Marathi New Year (Gudi Padwa), Telugu New Year (Yugadi) and Gujarati New Year starts with the new moon. Similarly, the first day of the year based on solar calendars, including Tamil New Year (Puthandu), Malayalam New Year, Bengali New Year (Pohela Boishakh or Naba Barsha) and Sikh New Year (Vaisakhi), starts when the sun transits from one zodiac to another i.e. on the important day of Sankranti.
In religious towns like Mathura and Vrindavan tourists throng temples on New Year eve. However this day has no religious significance according to Hindu calendar. If you miss it then you should not regret it as you can visit the temple on the next day.