Crazy trick I figured out. Fixed focal length camera for SLAM, so near field is blurred. But I need to track a fiducal. Apparently you can run a filter! Notice the crisp lines in the near field's fiducial. The camera is actually pointed at the second image!

The filter you need to run is really sensitive to the physical size of the tag and all of the camera intrinsics. I probably will just keep autofocus on for now for tracking it, but this will be a nice optimization later.