Maybe he cares a lot for you and is testing the waters to see how you will react.
This doesn't mean you need to fall into his arms declaring your undying love BUT things could be going to the next level.
OR he could be reeeeeeeally just joking around and thinks its funny. If this is the case I give my permission for you to slap him.
