img {
    border: solid 10px transparent;
}
img:hover {
    border-color: green;
}

Add a wrapper and pseudo-element

You cannot apply any style directly to the image that will give it an inset border on hover. But you can achieve the desired effect by wrapping each image in some kind of container element and adding an ::after pseudo-element with the applied border.

.border {
  display: inline-block;
  position: relative;
}
.border::after {
  content: '';
  position: absolute;
  top: 0;
  right: 0;
  bottom: 0;
  left: 0;
  box-shadow: inset 0 0 0 0 rgba(255,255,255,.5);
  transition: box-shadow .1s ease;
}
.border:hover::after {
  box-shadow: inset 0 0 0 10px rgba(255,255,255,.5);
}
img {
  display: block;
  position: relative;
}

<div class="border">
  <img src="http://placehold.it/150" />
</div>

You can use the :hover psuedo selector:

img:hover {
  border-color: red;
}

Also, if you want it to be like a button, consider these styles as well:

img {
  cursor: pointer;
}
img:active {
  border-color: blue;
}

img {
  border: 2px solid black;
  cursor: pointer;
}
img:hover {
  border-color: red;
}
img:active {
  border-color: blue;
}

<img src="http://via.placeholder.com/150x150" alt="">
<img src="http://via.placeholder.com/150x150" alt="">
<img src="http://via.placeholder.com/150x150" alt="">

And using box-shadow is just as simple:

img {
  border: 2px solid black;
  cursor: pointer;
}
img:hover {
  box-shadow: 0 0 6px black;
}
img:active {
  box-shadow: 0 0 6px blue;
}

<img src="http://via.placeholder.com/150x150" alt="">
<img src="http://via.placeholder.com/150x150" alt="">
<img src="http://via.placeholder.com/150x150" alt="">

and to make the image a bit bigger, use transform: scale(1.1);

img {
  border: 2px solid black;
  cursor: pointer;
}
img:hover {
  box-shadow: 0 0 6px black;
  transform: scale(1.1);
}
img:active {
  box-shadow: 0 0 6px blue;
}

<img src="http://via.placeholder.com/150x150" alt="">
<img src="http://via.placeholder.com/150x150" alt="">
<img src="http://via.placeholder.com/150x150" alt="">