The TV star, 26, who will be seen by viewers in camp from Wednesday's show, said: "Made In Chelsea is no longer challenging to me. I don't get the excitement that the new cast experience when they come in and film for the first time.
"I'll help them out. I'm not saying I won't, but I'm not looking to go back to Made In Chelsea as a cast member again.
"We're talking about all sorts of ideas, but I can't go into much detail at this moment. Hopefully, 2016 should be a good year for us."
Spencer, who is single after splitting from long term girlfriend Lauren Hutton in September, also admitted he fancied Jorgie Porter in camp.
He is taking in fluffy slippers he intends to let girls like Jorgie wear to win them over and said: "Obviously, Jorgie Porter is in the jungle and that's fun.
"Whenever I've watched Hollyoaks, I've always been attracted to her.
"Who wouldn't be? She is lovely. We also have quite a few mutual friends, which I didn't know before now.
"The girlfriend of one of my good mates is best friends with Jorgie Porter.
"I've tried not to look too much into the people in there before meeting them in person.
"One of the things that we face a lot is people being prejudiced.
"Having watched the show, people think they know you.
"There was a stage where I could've read up about everyone in there, but I didn't do any research on anyone really."
Spencer also insisted he would not go all the way in camp even if he does find love and played down his partying past.
He said: "I'll be absolutely fine. Everyone's got this idea that I'm a crazed sex addict, but that's not the case at all. I've gone that long before.
"Quite recently, actually. I'm single now, so I have sex far less now than I did when I was in a relationship. I'm also no longer 19, unfortunately – so going out every night and being promiscuous is not as normal as it once was for me.
"I think I need to at least pretend to have grown up a bit. I'm perfectly happy with my own company, which is a skill I've recently acquired. I'm looking forward to making new friends in there, but I don't think sex will be a major part of my frustration."
On a more serious note, he also told how he is following in the footsteps of his brother Michael who was an adventurer and the youngest man to climb Everest.
But he died aged 22 when Spencer was aged 10.
He added: "Whenever you find yourself in a tough period or tough spot I often do think of my brother and everything he went through and I suppose he will help me through."
Spencer Matthers: Love or hate?
0+ VOTES SO FAR
Think you know everything there is to know about Made in Chelsea? Take our fun interactive quiz and test your knowledge.