MILLBURY (CBS) — A substitute teacher at Elmwood Street School in Millbury was removed from the pre-K classroom he was working in after another teacher noticed he had a loaded magazine, police said.

Around 11 a.m. Thursday, a full-time teacher heard something hit the floor and noticed the substitute, a 22-year-old man, whose name is being withheld, pick up the magazine.

READ MORE: Boston Police Searching For Missing Girl In Mattapan

The teacher immediately notified administrators, who removed the man and called police.

The man confirmed to officers that he had ammunition with him in the classroom, police said.

He was escorted home by police and once there, officers found several rifles and shotguns, about half a dozen handguns, and several boxes of ammunition.

READ MORE: Plympton Couple Starring In New HGTV Show On Repairing Historic Homes

Police said, while he was properly licensed, he was charged and summonsed for improperly stored armed weapons. His license was also immediately suspended.

Parents learned about the incident through a phone call from Superintendent of Schools Greg Myers Thursday evening.

Myers confirmed to WBZ-TV that the man, who was working on a day-by-day contract was fired. He also said the incident appears to be an honest mistake: the man had been at the shooting range Wednesday and forgotten about the magazine in his jacket pocket.

“There were no charges for a live ammunition on school property, had he had a firearm with him then that would have been a violation. To the best of our knowledge, he did not have a firearm, it was strictly ammunition,” explained Police Chief Donald Desorcy.

MORE NEWS: 3 Massachusetts Counties Improve To 'Substantial' On CDC COVID Risk Map

On Friday morning, students were greeted by an extra police presence, including a K-9 unit, as a precaution.