Magnetic Resonance Imaging (MRI) is widely used in diagnosing anterior cruciate ligament (ACL) injuries due to its ability to provide detailed image data. However, existing deep learning approaches often overlook additional factors beyond the image itself. In this study, we aim to bridge this gap by exploring the relationship between ACL rupture and the bone morphology of the femur and tibia. Leveraging extensive clinical experience, we acknowledge the significance of this morphological data, which is not readily observed manually. To effectively incorporate this vital information, we introduce ACLNet, a novel model that combines the convolutional representation of MRI images with the transformer representation of bone morphological point clouds. This integration significantly enhances ACL injury predictions by leveraging both imaging and geometric data. Our methodology demonstrated an enhancement in diagnostic precision on the in-house dataset compared to image-only methods, elevating the accuracy from 87.59% to 92.57%. This strategy of utilizing implicitly relevant information to enhance performance holds promise for a variety of medical-related tasks.