For Elmo I used a red cupcake wrapper and did his fur with red frosting using the Wilton decorating tip #22 (from what I remember, but it may have been a little smaller of the same shape). I started on the outside of the cupcake and worked my way in, kind of in circles, pulling the frosting up as I went. I used a dab of black frosting and 1/2 marshmallows for his eyes and stuck a little frosting as well as cutting a little bit of the marshmallow off each side in the middle where they touched each other to keep them together. His mouth was 1/2 of the chocolate part of an Oreo cookie (cut in half carefully with a serrated bread knife) and his nose is an orange peanut butter m&m.
Cookie monster had a blue cupcake wrapper and blue frosting and I used Wilton decorating tip #134 to make his fur. I started on the outside with his fur and worked in, adding a little more after I stuck the Oreo on to cover it at the top. His eyes & mouth were done the same way as Elmo but I offset his eyes to make him look kooky :)
Oscar's fur was done the same way as Cookie Monster's but with green frosting. I was originally going to use a green cupcake wrapper for him but came up with the idea to use silver, so it would look like his trash can! His eyes are white m&m's with black frosting dots and a black frosting squiggle for his eyebrow (Wilton tip #1 for both). His mouth is the same Oreo cookie as Elmo & Cookie Monster.
Grover also got a blue cupcake wrapper and his fur was done the same way as Elmo's. His eyes are white m&m's with a dot of black frosting using Wilton decorating tip #1. His nose is a pink gobstopper, his mouth is 1/2 a chocolate Oreo cookie with some red frosting to accent it with Wilton decorating tip #1. If I'd had time I think I would have tried to look for some other candy to use for Grover's eyes, since they're not perfectly round but slightly oval.
For Ernie I used a red cupcake wrapper because he has red stripes on his shirt. I frosted his face smooth with orange icing. He's got the Oreo cookie mouth, white m&m's with a dot of black icing for his eyes, orange m&m's for his ears, a red peanut butter m&m for his nose and some squiggly black frosting for his hair (Wilton tip #2).
Here is Bert pre-mouth & nose. For him I kept the tops of the cupcakes mostly yellow and I used a green cupcake wrapper because he wears green pants. I frosted an oval for his face in yellow frosting and used white m&m's with black frosting dots for his eyes, yellow m&m's for his ears, an orange peanut m&m for his nose and the smile, hair and unibrow were done with black frosting (Wilton tip #2).
The Count got a green cupcake wrapper because his clothes are green. I frosted the whole top of the cupcake smooth with purple/dark lavender frosting. I used little pieces of marshmallows for his eyes, and Oreo cookie for his mouth, more purple frosting for his ears, nose and around his eyes (Wilton tip #3), white frosting for his fangs (I just frosted those by hand, but they would have looked better if I used tip #1), and black frosting for his hair, eyes, and goatee (Wilton tip #1). The Count took the most time to decorate because of all his small details.
I saved him for last because I thought Big Bird was going to be really really hard since I had never done frosting like his feathers but these actually weren't too bad. I used a pink wrapper because of the pink stripes Big Bird has on his legs and yellow frosting for his feathers. I used Wilton decorating tip #102, fanning each feather out from left to right as I worked around the whole cupcake. I started from the outside with circles of feathers and worked my way in. I made the feathers pretty big, so it only took about 4 rows all together. His eyes are white m&m's with a black frosting dot and a bright blue strip for his eyelids (I used Wilton tip #1 for those).
I made the top feathers and his beak out of candy melts tinted yellow. Because I didn't need much of the candy melts for this, I found it was easiest to use a zip lock bag with a (small!) hole cut in the corner for the melted candy melts instead of a decorating tip with a coupler. I used two pieces for his beak, with the top one being bigger than the bottom party. His tongue is red frosting.