You can use the before or after pseudo-element and apply some CSS to it. There are various ways. You can add both before and after, and rotate and position each of them to form one of the bars. An easier solution is adding two borders to just the before element and rotate it using transform: rotate.

Scroll down for a different solution that uses an actual element instead of the pseuso elements

In this case, I've added the arrows as bullets in a list and used em sizes to make them size properly with the font of the list.

ul {
    list-style: none;
}

ul.big {
    list-style: none;
    font-size: 300%
}

li::before {
    position: relative;
    /* top: 3pt; Uncomment this to lower the icons as requested in comments*/
    content: "";
    display: inline-block;
    /* By using an em scale, the arrows will size with the font */
    width: 0.4em;
    height: 0.4em;
    border-right: 0.2em solid black;
    border-top: 0.2em solid black;
    transform: rotate(45deg);
    margin-right: 0.5em;
}

/* Change color */
li:hover {
  color: red; /* For the text */
}
li:hover::before {
  border-color: red; /* For the arrow (which is a border) */
}

<ul>
    <li>Item1</li>
    <li>Item2</li>
    <li>Item3</li>
    <li>Item4</li>
</ul>

<ul class="big">
    <li>Item1</li>
    <li>Item2</li>
    <li>Item3</li>
    <li>Item4</li>
</ul>

Of course you don't need to use before or after, you can apply the same trick to a normal element as well. For the list above it is convenient, because you don't need additional markup. But sometimes you may want (or need) the markup anyway. You can use a div or span for that, and I've even seen people even recycle the i element for 'icons'. So that markup could look like below. Whether using <i> for this is right is debatable, but you can use span for this as well to be on the safe side.

/* Default icon formatting */
i {
  display: inline-block;
  font-style: normal;
  position: relative;
}

/* Additional formatting for arrow icon */
i.arrow {
    /* top: 2pt; Uncomment this to lower the icons as requested in comments*/
    width: 0.4em;
    height: 0.4em;
    border-right: 0.2em solid black;
    border-top: 0.2em solid black;
    transform: rotate(45deg);
}

And so you can have an <i class="arrow" title="arrow icon"></i> in your text.
This arrow is <i class="arrow" title="arrow icon"></i> used to be deliberately lowered slightly on request.
I removed that for the general public <i class="arrow" title="arrow icon"></i> but you can uncomment the line with 'top' <i class="arrow" title="arrow icon"></i> to restore that effect.

If you seek more inspiration, make sure to check out this awesome library of pure CSS icons by Nicolas Gallagher. :)

Use '›'

› -> single right angle quote. For single left angle quote, use ‹

Unicode arrows heads:

• ▲ - U+25B2 BLACK UP-POINTING TRIANGLE
• ▼ - U+25BC BLACK DOWN-POINTING TRIANGLE
• ▴ - U+25B4 SMALL BLACK UP-POINTING TRIANGLE
• ▾ - U+25BE SMALL BLACK DOWN-POINTING TRIANGLE

For ▲ and ▼ use ▲ and ▼ respectively if you cannot include Unicode characters directly (use UTF-8!).

Note that the font support for the smaller versions is not as good. Better to use the large versions in smaller font.

More Unicode arrows are at:

• http://en.wikipedia.org/wiki/Arrow_%28symbol%29#Arrows_in_Unicode
• http://en.wikipedia.org/wiki/Geometric_Shapes

Lastly, these arrows are not ASCII, including ↑ and ↓: they are Unicode.

p:before { content:"\2039"; }
p:after  { content:"\203A"; }

A different solution for this particular example (sans rotating and using chevron character codes)

The short answer to your question is "No, there aren't."

&laquo and &raquo are not actually intended as direction arrows, despite their frequent usage as such.

What they are actually intended for is hinted at by their entity names -- they're supposed to be "angled quotes".

Certain European countries (notably France) tend to use these characters for speech marks instead of the inverted-comma style quote marks that are more commonly used by the English speaking world, and that is what these characters are actually meant for -- see http://en.wikipedia.org/wiki/Guillemet for more info.

Given this, it's hardly surprising that there aren't any up and down arrow variants of them.

Feel free to use them as directional arrows (everyone else does, even if the semantics are questionable), but don't expect them to have every variant you might want, because they're not graphics; they're typographical characters.

I guess if you really want to, you could use CSS to rotate the existing left and right arrows by 90 degrees. I wouldn't suggest it as a good idea though.

To make the selected a tag has full border, I updated ul.chevrons li a.selected class. In this class, I added height to make the selected a has same height with other a tags. Then I added new classes top-border and bottom-botter and removed the gap created by :after and :before.

ul.chevrons li a.selected {
  border: 1.5px solid yellow;
  background-color: #013e75;
  color: #fff;
  height: calc(2.5em - 3px);
}

const selectedLink = document.querySelector('.chevrons li a.selected');

const topBorder = document.createElement('span');
topBorder.classList.add('top-border');

const bottomBorder = document.createElement('span');
bottomBorder.classList.add('bottom-border');

selectedLink.appendChild(topBorder);
selectedLink.appendChild(bottomBorder);

html {
  font-family: sans-serif;
  font-size: 1em;
  -ms-text-size-adjust: 1em;
  -webkit-text-size-adjust: 1em;
  -webkit-tap-highlight-color: rgba(0, 0, 0, 0);
}

body {
  margin: 0;
  font-family: Arial, 'Helvetica Neue', Helvetica, sans-serif;
  line-height: 1.42857143;
  color: #333333;
  background-color: #ffffff;
}

#wrapper {
  width: 600px;
  margin: 50px auto;
}

ul.chevrons {
  padding: 0;
  margin: 0;
  height: 2.5em;
  font-size: 1.125em;
  background-color: #e3edf9;
  overflow: hidden;
  width: 100%;
}

ul.chevrons li {
  list-style: none;
  text-align: center;
  line-height: 2.5em;
  float: left;
}

ul.chevrons li {
  width: 25%;
  /* This changes based on how many items are in the flow. 3 items = 33% and 2 items = 50% */
}

ul.chevrons li a {
  width: 100%;
  color: #212121;
  text-decoration: none;
  display: inline-block;
  position: relative;
  text-indent: 0.75em;
}

ul.chevrons li a.selected {
  border: 1.5px solid yellow;
  background-color: #013e75;
  color: #fff;
  height: calc(2.5em - 3px);
}

ul.chevrons li a.previous {
  color: #6c6c6c;
}

ul.chevrons li a:before,
ul.chevrons li a:after {
  content: " ";
  display: block;
  width: 0;
  height: 0;
  border-top: 1.5em solid transparent;
  border-bottom: 1.5em solid transparent;
  position: absolute;
  top: 50%;
  margin-top: -1.5em;
  left: 100%;
}

ul.chevrons li a:before {
  border-left: 0.75em solid yellow;
  margin-left: -0px;
  z-index: 1;
}

ul.chevrons li a:after {
  border-left: 0.75em solid #e3edf9;
  margin-left: -2px;
  z-index: 2;
}

ul.chevrons li a.selected:after {
  border-left: 0.75em solid #013e75;
}

ul.chevrons li:last-child a:before,
ul.chevrons li:last-child a:after {
  display: none;
}

.top-border {
  position: absolute;
  top: -1.5px;
  width: 4px;
  height: 1.5px;
  background: yellow;
  right: -2px;
  z-index: 1000;
}

.bottom-border {
  position: absolute;
  bottom: -1.5px;
  width: 4px;
  height: 1.5px;
  background: yellow;
  right: -2px;
  z-index: 1000;
}

<div id="wrapper">
  <ul class="chevrons">
    <li><a href="#" class="previous">Step 1</a></li>
    <li><a href="#" class="selected">Step 2</a></li>
    <li><a href="#">Step 3</a></li>
    <li><a href="#">Step 4</a></li>
  </ul>
</div>