A right-sided midfielder who was strong in tackle, had a good engine and was dangerous in attack. He was already an experienced player before arriving at Anfield, having played 101 league games for Tranmere in Third Division North and from 1951 to 1956, 205 games for First Division Bolton. Wheeler played in the famous "Matthews final" where Bolton lost 3-4 after leading 3-1 in the 1953 FA Cup final due to Blackpool's winger Stanley Matthews' brilliance on the day. Liverpool manager Phil Taylor put Wheeler's wealth of experience to good use as Liverpool tried to get promoted from Second Division and made Wheeler Liverpool's captain in the 1958-59 season.
Wheeler was a versatile player who normally played at wing-half but many of his early appearances for Liverpool came as an inside-forward, including the remarkable day against Port Vale early in November 1956 when, after Billy Liddell had equalised Vale's first-half goal, he scored one of the fastest hat-tricks ever recorded, his goals coming in the 81st, 82nd and 85th minutes. Johnny was a regular in the side between, making a total of 108 Second Division appearances from the maximum during that period of 126.
During the 1960-61 season he had to share the No. 4 shirt with the young Gordon Milne. By the start of the following season he was out of the first-team picture altogether and was only selected for one more League game, covering for left-half Tommy Leishman when Plymouth Argyle visited Anfield on 9th December 1961.
He was appointed player-manager of New Brighton in May 1963, but did not take up the post, becoming assistant trainer at Bury instead where he stayed until 1969.