In response to the Judeans asking Jesus to reveal Himself, Jesus talked about how despite telling him, they did not believe. But in the conversation, Jesus extends the sheep analogy.
Jesus talks about how the questioners do not believe "because [they] are not of [His] sheep" (10:26). In the context, being a "sheep" means being someone who was receptive of God, especially since the sheep was aware of what the voice sounds like before the voice was called in the first place as mentioned in 10:4-5. Although the questioners may be familiar with God and even the Mosaic Law, they did not truly understand God (especially since past Scriptures hint about Jesus like in Isaiah).
In contrast, Jesus's sheep "hear [His] voice" and "follow [Him]" (10:27). As the phrasing is part of an analogy (and a pre-cursor to receiving everlasting life as stated shortly after), the "following" is obeying his call to salvation (via believing in Jesus). Ever believer has followed Jesus at some point because they obeyed Jesus's call to believe in Jesus.