Goals from Matt Mills and Apostolos Vellios had turned the game around following Jon Taylor's first-half opener and the points looked to be heading Forest's way after Rotherham's Darnell Fisher had been sent off.
To their credit The Millers rallied and Taylor equalised with three minutes remaining as Forest were left to settle for their second straight 2-2 draw and a fifth game without defeat in all competitions.
Head coach Philippe Montanier made four changes from the weekend draw at Aston Villa. Nicolao Dumitru came in for Mustapha Carayol with Jorge Grant replacing the injured Matty Cash. In defence Matt Mills replaced Michael Mancienne while Armand Traore took his place in the starting line-up ahead of Danny Fox.
The opening quarter of an hour passed with both teams struggling to find any real rhythm to their play. The home side posed a threat on the right-hand side through Fisher and Taylor without creating any real goalscoring chances.
However it was the visitors who were the first to show as they tested Lee Camp in the Rotherham goal. After 18 minutes Grant found himself in space outside the area only to see his curling effort go straight into the arms of the former Forest keeper.
Three minutes later Forest created a second chance when Ben Osborn brought down a high ball from Lichaj, and after side stepping Wilson he managed to pull the ball back to Vellios whose shot cleared the cross bar. It was difficult opportunity for the Greek international from a tight angle.
However it was the hosts who drew first blood when, on 29 minutes, midfielder Taylor finished off an impressive counter-attack which caught Forest on the back foot.
Izzy Brown broke through the Forest defence with a powerful run from the halfway line and, after Perquis and Lichaj managed to regroup, Brown found Taylor unmarked in the area and he calmly slotted the ball into the Forest net to give the home side the advantage.
Ten minutes later Vellios nearly put Forest level. He managed to skip past Wilson and fire a shot on the half-volley only to see Camp get down well and turn the ball around the post.
After the interval Forest replaced Perquis with Mancienne and then on the hour mark, Nicklas Bendtner came on for his debut in place of Dumitru.
Shortly after the introduction of Bendtner, Forest nearly levelled up the contest. Lansbury was inches away from a ball curled in by Grant from the right-hand channel and after Osborn turned the ball back in play, it was Kasami who failed to keep his shot down from inside the Rotherham area.
The visitors seemed to grow in confidence after that opportunity and found themselves level after 76 minutes. The corner came in from Lansbury on the right and found Mills, the defender found some space in the area and directed his header into the bottom left-hand corner of Camp’s goal.
On 83 minutes Fisher was shown a second yellow card, thus reducing the hosts to ten men and from the subsequent free kick Forest found themselves in front.
Lansbury’s cross again found Mills at the back post, he headed back across the face of goal and after a neat flick from Bendtner, Vellios was on hand to fire home a spectacular scissor volley at the back post and score his second goal in as many games.
Just when it looked like it would be the visitors who would take home the three points Rotherham somehow managed to find an equaliser on 87 minutes through a second goal from Taylor. The midfielder came inside from the left and after taking on Lichaj and Mills powered the ball past Stojkovic.
The final chance fell to Forest almost straight from the restart. Lichaj put in an excellent cross, Lansbury was denied by a stunning stop from Camp before Bendtner headed on to the post.
It was to be the last action of a thrilling night at the New York Stadium as Forest were forced to take a 2-2 draw back to Nottingham but it is now five games without defeat for Philippe Montanier's side ahead of Saturday's clash with Norwich City.
Forest: Stojkovic, Lichaj, Mills, Perquis (Mancienne 46'), Traore, Dumitru (Bendtner 59'), Lansbury (c), Grant (Licá 84'), Kasami, Osborn, Vellios
Subs not used: Smith, Fox, Carayol, Ahmedhodzic
Goals: Mills 76', Vellios 85'
Bookings: Osborn 57', Lansbury 64', Lichaj 90'
Rotherham: Camp, Mattock, Ward (Yates 69' (Kelly 84')), Taylor, Halford, Wilson, Fisher, Fry, Allan, Adeyemi (Frecklington 82'), Brown
Subs not used: Price, Vaulks, Forde, Ball
Goals: Taylor 30', 87'
Bookings: Mattock 89', Brown 89'
Sent off: Fisher 83'
Referee: Peter Bankes
Attendance: 10,243 (2,138 away)