Krishna makes a plain, practical distinction. For the person who has not disciplined the mind, yoga is indeed hard to reach. He does not sugarcoat it. But for the person who strives with a controlled mind and uses the right approach — upayatah, through proper means — yoga is within reach.
That word 'upayatah' carries weight. It means 'through the right method.' Brute willpower alone is not enough. A farmer does not grow crops by sheer determination. He needs the right seeds, the right soil, the right season. Similarly, the seeker needs the right practice, the right guidance, the right inner attitude. With these, the door stays open.
Krishna frames this as his personal view — 'iti me matih.' There is something almost conversational about it, as if one friend is sharing honest counsel with another.