“The house” was, perhaps, the home of Matthew. Don’t be too hard on the Pharisees for not being able to understand the ministry of Jesus and the callings He placed on the lives around Him. John wondered about the same things (Matthew 11:2-3).
When Jesus doesn’t give us details about what He is calling us to until after we surrender to the call, we wonder many things, too. We also wonder about the validity of each others’ callings when they don’t make sense.
The answers to our questions for clarity and validity will always come from Jesus. Modern Pharisees, or even modern leaders like John who have not received revelation about some new thing Jesus is doing will always persecute what comes next. Don’t join them. Ask Jesus what He thinks about it. Join Him.