Jo Midori wasn't just helping; she was his primary teacher in the samurai way. The lessons learned afterwards at the palace were just a fine tuning of the stuff she had taught him as her student. So in a way he considers her his mentor, if theres any real idolizing, it'd be there. But mild, I assure you. So I'm sure her name was mentioned at least once or twice, and stories would also be said of things Jo Midori had learned from her teachers, no?
Another thing to keep in mind too, is that because of Sumaru's outward appearance and previously Yamataian self, he's probably dealt with much scrutiny and patronizing all throughout his process of becoming a samurai. So being with a experienced samurai, no matter who they are really, and considered a equal. Or at least something of the sort, is something he finds to be well, for the lack of a better word at the moment; great!
Though you may be seeing it as idolizing, Sumaru is a person who highly regards anyone with seniority in the fields he is a part of. Just like many people in Japanese culture, with the need for honorifics to one's seniors so stressed in use. Sort of a way of acknowledging that they have more experience than him and are someone worth learning from. It's really not idol worship at all, even his gratitude towards Sakai Yuuko isn't really idol warship, it's all just a understanding that he is always learning. So, who better to learn from, then someone who's been there before him and has more lessons learned then himself. In that very same sense, he approaches most people more experienced than him in the same humble manner.