High-resolution structural designs attracts researchers to multi-scale topology optimizations (TO) paradigms. With the advances of machine learning (ML) methods, the integration of ML with TO has been attempted in many works. However, most works employ ML in a data-driven paradigm, which requires abundant training data. The generalization ability of such a data-driven paradigm is also ambiguous. This research aims to utilize the machine learning techniques as an optimizer for multi-scale structural design problems to address the connectivity issues of adjacent microstructures, a common problem in the multi-scale structure design. First, parameterized cellular materials (PCM) are utilized to develop a multi-scale parameterized TO problem. Then the problem is reformulated into a single unconstrained objective function using the penalty method and parameterized into a neural network (NN) that optimizes its weights and biases. The optimized network acts as a continuous model all over the design domain with the cellular material parameter as its response. This approach does not need to eliminate the elements with the intermediate densities, unlike density-based TO frameworks (e.g., SIMP). Using the NN-assisted optimizer, to handle the connectivity issue, the optimized NN can be discretized to a higher resolution, eliminating the need to use an interpolation filter. The performance of the proposed framework is significantly enhanced compared to the previously published method.