Bug report

Bug summary
Rasterizing the marker collection results in a large file size as compared to rasterizing the whole axes. See the example below:

Code for reproduction

import matplotlib.pyplot as plt
import numpy as np

data = np.random.rand(100, 100)

fig1, ax1 = plt.subplots()
ax1.plot(data, 'o', rasterized=True)
fig1.savefig('rasterizing_markers.pdf')

fig2, ax2 = plt.subplots()
ax2.set_rasterized(True)
ax2.plot(data, 'o')
fig2.savefig('rasterizing_axes.pdf')

Actual outcome
rasterizing_axes.pdf is around 123 KB and rasterizing_markers.pdf is around 447 KB! (the only difference in those two files is that rasterizing_markers.pdf has spines, ticks and tick labels in vector format)

Expected outcome
Expecting that both files will have similar size. I am sure that the difference is not because of the embedded fonts (If I change data = np.random.rand(100, 100) to data = np.random.rand(10, 10), the file size for rasterizing_markers.pdf is 23 KB so that rules out the role of embedded fonts in explaining the ~300 KB difference).

Matplotlib version

• Operating system: MacOS
• Matplotlib version: 3.0.3
• Matplotlib backend (print(matplotlib.get_backend())):
• Python version: 3.7
• Jupyter version (if applicable):
• Other libraries: