The hole in the frame on the CZ 85, right side, is larger in diameter than the hole on the right side of a CZ 75.
There is an internal portion of the frame (to the left side of the spring/groove that is used to hold the two slide release pieces in the frame that may be smaller near the left side/inside of it. I don't recall, in all the times I had mine apart, every looking to see if that inside hole was stepped (smaller on the left, to support the left side slide release) or if it was the same diameter all the way through (same as the hole in the right side of the frame).
Even if that hole is stepped (smaller for part of it) and you install a CZ 75 slide release, it will allow more dirt/dust/crud into the frame.
Any reason not to trade the CZ 85 in on a CZ 75? You can get CZ 75's with ambidextrous safeties, too.
If you've just got to keep the CZ 85, have you thought about removing the right side slide release and filing/sanding it smooth, to get rid of the grooves? It would have to be refinished, of course.
I haven't noticed my right side slide release damaging the holsters but I've sure noticed the holsters wearing the finish off the right side slide release (minor problem).