Globally new pandemic diseases induce urgent demands for portable diagnostic systems to prevent and control infectious diseases. Smartphone‐based portable diagnostic devices are significantly efficient tools to user‐friendly connect personalized health conditions and collect valuable optical information for rapid diagnosis and biomedical research through at‐home screening. Deep learning algorithms for portable microscopes also help to enhance diagnostic accuracy by reducing the imaging resolution gap between benchtop and portable microscopes. This review highlighted recent progress and continued efforts in a smartphone‐tethered optical platform through portable, automated, and deep‐learning‐enabled microscopy for personalized diagnostics and remote monitoring. In detail, the optical platforms through smartphone‐based microscopes and lens‐free holographic microscopy are introduced, and deep learning‐based portable microscopic imaging is explained to improve the image resolution and accuracy of diagnostics. The challenges and prospects of portable optical systems with microfluidic channels and a compact microscope to screen COVID‐19 in the current pandemic are also discussed. It has been believed that this review offers a novel guide for rapid diagnosis, biomedical imaging, and digital healthcare with low cost and portability.