Fisheye camera calibration is an essential task in photogrammetry. However, previous calibration patterns and the robustness of the adjoint processing methods are limited due to the fisheye distortion and various lighting. This problem leads to additional manual intervention in the data collection. Moreover, it is arduous to accurately detect the board target under fisheye's distortion. To increase the robustness in this task, we present a novel encoded board “Meta‐Board” and a learning‐based target detection method. Additionally, an automatic image orthorectification is integrated to alleviate the distortion effect on the target iteratively until convergence. A low‐cost control field with the proposed boards is built for the experiment. Results on both virtual and real camera lenses and multi‐camera rigs show that our method can be robustly used in calibrating the fisheye camera and reaches state‐of‐the‐art accuracy.