Brad Paisley made a surprise appearance on American Idol to pull a few pranks during the show’s season premiere on Monday night (January 26).

Paisley recruited judge Luke Bryan for a prank on the show. Bryan excused himself from the audition room in Nashville to use the restroom. While fellow judges Lionel Richie and Carrie Underwood got ready for the show, Underwood pointed out a sign reading “Brad Paisley and Kimberly Williams-Paisley Ballroom.”

“Oh, I can’t wait to ruin this whole show,” Paisley said as he met up with Bryan in a room set up as a backstage area. Bryan replied, “we’re counting on ya.”

Bryan wore an earpiece, so Paisley could give him “ridiculously awkward stuff” to say during an audition. Fake auditioner Dylan Holmes introduced himself to Paisley and Bryan before they teamed up to pull the prank. Paisley and Bryan agreed a “safe word” would be “hot chicken,” and Paisley told the Idol judge, “don’t chicken out! Don’t hot chicken out!”

Bryan returned from the “restroom,” and took his seat next to Underwood. Paisley started by prompting him to ask the “Before He Cheats” star, “hey, Carrie, how much do you think I could bench press?”

Holmes, 26, entered the audition room, and shared that he’s from Michigan. Bryan exclaimed, “I can’t stand Michigan!”

“What is happening?” Underwood wondered. Richie said, “oh, that’s it. …We just lost all of Michigan.”

“PASS!” Paisley prompted Bryan to shout after Holmes strummed only a few notes on his guitar. Next, he prompted Bryan to request Holmes play a song by Richie.

“I need more hot chicken,” Bryan said after Holmes’ audition. That’s when Paisley entered the room with host Ryan Seacrest, and revealed to Richie and Underwood that it had been a prank all along. Underwood said during a recent appearance on Jimmy Kimmel Live that “the worst part about that, though, he [Bryan] was acting really weird, but I really didn’t think that much of it. You know what I mean? I was like, ‘what are you doing?’ But it wasn’t that ‘off.’”

Watch Paisley and Bryan team up to pull pranks here: