You've applied to the body, that basically means the whole page. Insert a DIV on the body.

HTML

<div class="div-class"></div>

CSS

.div-class{
  background-color: yellow;
  border: 1px solid #000000;
  width: 300px;
  height: 300px;
}

Most of the currently accepted answer is inaccurate:

You can change the default border color: not by CSS, but in the user's graphic environment (system settings, usually available as desktop settings in OS).

You can omit the color value in border shorthand property. In CSS3 the border-color is then set to currentColor, which can also be specified explicitly.

border: 1px solid currentColor; /* CSS3 */

The currentColor is usually black by default system settings. In CSS2, you can also use other system values, see in the link above. These are deprecated, but still working in my Opera.

border: 1px solid ThreeDDarkShadow; /* CSS2 deprecated */

Now the color is gray in my environment. The CSS2 values are (quoting the link above):

ActiveBorder, ActiveCaption, AppWorkspace, Background, ButtonFace, ButtonHighlight, ButtonShadow, ButtonText, CaptionText, GrayText, Highlight, HighlightText, InactiveBorder, InactiveCaption, InactiveCaptionText, InfoBackground, InfoText, Menu, MenuText, Scrollbar, ThreeDDarkShadow, ThreeDFace, ThreeDHighlight, ThreeDLightShadow, ThreeDShadow, Window, WindowFrame, WindowText.

Note: currentColor is different from inherit (which will solve your problem as Quentin suggests) and there is no value keyword like default, auto or initial in border-color property. One might think that if you specify invalid or browser-unsupported color, the browser has to pick some color and if there is no way to infer that color, it logically picks the system color anyway since browsers don't stop output on syntax error. However, some browsers implement a mystical numerologic algorithm to infer colors from unknown strings. It doesn't apply in my Opera.

Check your system colors in the snippet

  div { float: left; margin: 5px; width: 125px; padding: 20px; border: 1px solid black; color: #800; text-shadow: 0 1px black;}
  .ActiveBorder { background-color: ActiveBorder; }
  .ActiveCaption { background-color: ActiveCaption; }
  .AppWorkspace { background-color: AppWorkspace; }
  .Background { background-color: Background; }
  .ButtonFace { background-color: ButtonFace; }
  .ButtonHighlight { background-color: ButtonHighlight; }
  .ButtonShadow { background-color: ButtonShadow; }
  .ButtonText { background-color: ButtonText; }
  .CaptionText { background-color: CaptionText; }
  .GrayText { background-color: GrayText; }
  .Highlight { background-color: Highlight; }
  .HighlightText { background-color: HighlightText; }
  .InactiveBorder { background-color: InactiveBorder; }
  .InactiveCaption { background-color: InactiveCaption; }
  .InactiveCaptionText { background-color: InactiveCaptionText; }
  .InfoBackground { background-color: InfoBackground; }
  .InfoText { background-color: InfoText; }
  .Menu { background-color: Menu; }
  .MenuText { background-color: MenuText; }
  .Scrollbar { background-color: Scrollbar; }
  .ThreeDDarkShadow { background-color: ThreeDDarkShadow; }
  .ThreeDFace { background-color: ThreeDFace; }
  .ThreeDHighlight { background-color: ThreeDHighlight; }
  .ThreeDLightShadow { background-color: ThreeDLightShadow; }
  .ThreeDShadow { background-color: ThreeDShadow; }
  .Window { background-color: Window; }
  .WindowFrame { background-color: WindowFrame; }
  .WindowText { background-color: WindowText; }

<div class="ActiveBorder">ActiveBorder</div>
<div class="ActiveCaption">ActiveCaption</div>
<div class="AppWorkspace">AppWorkspace</div>
<div class="Background">Background</div>
<div class="ButtonFace">ButtonFace</div>
<div class="ButtonHighlight">ButtonHighlight</div>
<div class="ButtonShadow">ButtonShadow</div>
<div class="ButtonText">ButtonText</div>
<div class="CaptionText">CaptionText</div>
<div class="GrayText">GrayText</div>
<div class="Highlight">Highlight</div>
<div class="HighlightText">HighlightText</div>
<div class="InactiveBorder">InactiveBorder</div>
<div class="InactiveCaption">InactiveCaption</div>
<div class="InactiveCaptionText">InactiveCaptionText</div>
<div class="InfoBackground">InfoBackground</div>
<div class="InfoText">InfoText</div>
<div class="Menu">Menu</div>
<div class="MenuText">MenuText</div>
<div class="Scrollbar">Scrollbar</div>
<div class="ThreeDDarkShadow">ThreeDDarkShadow</div>
<div class="ThreeDFace">ThreeDFace</div>
<div class="ThreeDHighlight">ThreeDHighlight</div>
<div class="ThreeDLightShadow">ThreeDLightShadow</div>
<div class="ThreeDShadow">ThreeDShadow</div>
<div class="ThreeDShadow">ThreeDShadow</div>
<div class="Window">Window</div>
<div class="WindowFrame">WindowFrame</div>
<div class="WindowText">WindowText</div>