We specifically do not want to use date2num in these test cases. the fact that it is erroring when handed datetime objects is precisely the kind of thing we wanted to find out by implementing these tests.

The following patch gets most of the way to fixing it such that you can actually pass datetime objects (though I think it may need some consideration regarding what to do if a non-data transform is provided, or whether "slope" is something that can be meaningfully provided with unitful data):

diff --git a/lib/matplotlib/axes/_axes.py b/lib/matplotlib/axes/_axes.py
index 9199f1cafa..524c1ae06a 100644
--- a/lib/matplotlib/axes/_axes.py
+++ b/lib/matplotlib/axes/_axes.py
@@ -919,6 +919,9 @@ class Axes(_AxesBase):
                                   self.get_yscale() != 'linear'):
             raise TypeError("'slope' cannot be used with non-linear scales")
 
+        xy1 = tuple(self._process_unit_info([("x", xy1[0]), ("y", xy1[1])]))
+        if xy2 is not None:
+            xy2 = tuple(self._process_unit_info([("x", xy2[0]), ("y", xy2[1])]))
         datalim = [xy1] if xy2 is None else [xy1, xy2]
         if "transform" in kwargs:
             # if a transform is passed (i.e. line points not in data space),

I did the minimal change here to get the point-point version working with the default transform (data space). I do think that the final version would need at least some more.

• passing kwargs in
• Maybe pushing the unit transforming down into mlines.AxLine? (trying to trace how base Line2D does it, if that does it in the class, I'd probably follow suit for AxLine, but not super clear on that, actually... AxLine inherits from Line2D)
• some more thought about slope may be good... it is already disallowed with non-linear scales, may be best to just disallow it with unitful data or declare it to be in explicitly "unitless data space" (currently it states that it is in 'data coordinates'). I think it may actually work out for at least the case of having the same units on both axes (i.e. the slope is unitless then) or possibly also with full unit libraries, though possibly would require pushing units into the AxLine class for it to work.