With the developments of ITS technology, it is considered that the V2V communication is necessary for the new kinds of applications in the future. However, there are actually some subjects of wireless networks between vehicles caused by the fast movements or the radio noise of the moving vehicles. Thus, this paper proposes the Delay Tolerant Network System with the Adaptive Array Antenna controlled by the image recognition for the V2V Networks. In the proposed system, the target vehicle is recognized by the Machine Learning based image recognition system, the Kalman Filter algorithm to modify the influence of the vehicle’s speed or the obstacles in the way of the road controls the direction of the Adaptive Array Antenna. The paper especially deals with the implemented image recognition system and the antenna direction controls from the experimental results of the prototype system, and the results indicate the effectiveness of the proposed system for the V2V networks.