I honestly think this is an incredibly subjective conversation, with the biggest discriminator being what you define as "safer".
If you define "safer" as being less likely to crash or get injured, then no - I do not believe riding on the track is safer than riding on the street. To the contrary, by that definition, I think riding on the street is probably safer - which would be factually proven by the "per capita" accident rate of street riding (by mile) compared to track riding. In other words, you're chances of crashing at the track are "probably" higher than your chances of crashing on the street - mile for mile.
However, if you define "safer" as your risk to have a serious or fatal injury (which is how I look at it) then I strongly believe the track is exponentially safer than the street. Think about all of the track days we've all been at - and frankly - even race days (whether we're racing or watching). How many fatalities have you seen in all of those crashes? How many traumatic brain injuries? The combination of well designed tracks, run-offs, air fences, leathers, helmets, full safety gear, the lack of intersections, careful observation of rules and procedures, FOCUS ON PURE RIDING, etc, all combine to allow us to have an off and walk away from it.
As somebody who has been riding for 45 years and who has crashed on the street, motocross tracks, and on the road track, I'm OK with the idea that I might get injured - believing that I'm less at risk for TBI, spinal injury or death - due to some big mac eating texting 17 year old kid pulling out in front of me.