Wordorigins.org

While the phrase twenty-three skidoo, meaning to go away, to leave, is associated with 1920s, it is actually somewhat older, dating to the turn of the 20th century. And the constituent element are even somewhat older.

Twenty-three is the oldest portion of the phrase. From the Morning Herald (Kentucky) of 17 March 1899:

For some time past there has been going the rounds of the men about town the slang phrase “Twenty-three.” The meaning attached to it is to “move on,” “get out,” “goody-bye, glad you are gone,” “your move” and so on. To the initiated it is used with effect in a jocular manner.