To the contrary, this is why
PureComponent exists for class components as well as
React.memo for functional components. It absolutely can be a performance issue, depending on the component itself — how often it re-renders, how expensive a re-render is, and whether or not the reference prop is the sole reason for the re-render.
The new Hooks API introduced a second parameter to
useCallback specifically so that the Hooks API does not break the performance benefits of blocking re-rendering for pure components.