Try typing 'E' in Kunstler Script in MS Word. IGCSE Math papers in a particular syllabus uses that.

The use of or is either from papers and books (and sometimes people) predating modern typesetting standards, or wrong (in term of standard notation). Or, of course, there is a confusion between the symbols because of the striking similarities between all of them.

The symbol and its variant are derived from the Norwegian-Danish letter Ø:

Common notations for the empty set include "{}", "Ø", and "\emptyset". The latter two symbols were introduced by the Bourbaki group (specifically André Weil) in 1939, inspired by the letter Ø in the Norwegian and Danish alphabet (and not related in any way to the Greek letter Φ).[1] Other notations for the empty set include "Λ" and "0".[2]

The empty-set symbol ∅ is found at Unicode point U+2205.[3] In TeX, it is coded as \emptyset or \varnothing.

(Wikipedia, Empty set)

As for universal sets, those are practically non-existent in modern set theory. I have seen them denoted by or sometimes , but it might as well be any other letter.

The important thing is that the reader understand what each symbol means. Of course, if there are standard symbols it is better to use them. In the case of the empty set there is. In the case of a universal set, there isn't really a standard notation for it.

I am not sure what you want to achieve, but I think you want to place the label for the universal set at a place where it does not intersect with the surrounding box.

This answer explains how you can use the macro \setpostvennhook to customize the position of the labels. The manual of the package further adds that there are coordinates in all four corners of the venn diagram, which you can use to align the labels. You can also use the options vgap or hgap to increase the margin between the circles and the outer box.

Of course, if the contents of the label is too long, it will eventually "crash" into the right border of the surrounding box ...

\documentclass[border=10pt]{standalone}
\usepackage{tikz}
\usepackage{venndiagram}

\begin{document}

\begin{venndiagram3sets}[labelOnlyA={1}, labelOnlyB={2}, labelOnlyC={}, labelOnlyAB= {}, labelOnlyAC={}, labelOnlyBC={9}, labelABC={3}, vgap=.75cm]

    \setpostvennhook{
        \node[anchor=south west] at (venn bottom left) {$\xi$ \{4,5,6,7,8,10\}};
    }

\end{venndiagram3sets}

\end{document}   

The correct command to use in this case is \bigcup; you could go with

$\bigcup\limits_{i=1}^{\infty} F_{i}$

but this will irremediably spoil your document. When a formula is inline, it's better if limits are set to the side of the symbol. In display math mode the \limits is implicit. See the following example and judge for yourself.

\documentclass{article}
\begin{document}

The correct command to use in this case is \verb|\bigcup|;
if you use \verb|\limits| your formula will appear inline
like $\bigcup\limits_{i=1}^{\infty} F_{i}$, but this will
irremediably spoil your document. When a formula is inline,
it's better if limits are set to the side of the symbol. In
display math mode the \verb|\limits| is implicit. Here is the
same formula without it $\bigcup_{i=1}^{\infty} F_{i}$ and you
can clearly see that the vertical spacing is much better.
In a display there is no problem
\[
\bigcup_{i=1}^{\infty} F_{i}
\]
and the limits can be set above and below without spoiling
the spacing, which is an important factor when readability
is considered.

\end{document}

To answer your specific question:

To quote from Leslie Lamport's "LaTeX: A Document Preparation System" (Leslie Lamport initially developed LaTeX):

The ten special characters

# $ % & ~ _ ^ \ { }

are used only in LaTeX commands.

...

Seven of those symbols can be produced by typing a \ in front of the corresponding character.

$ & % # _ { } are easy to produce.

In other words, to get { and } you must type \{ and \}. This is because { and } mean something very special to LaTeX. They are used to delimit arguments. So with something like \emph{Hi}, the emphasis starts at the { and finishes at the } (\emph is a command which is used to emphasise text; the standard behaviour is for it to italicise it). In other words, everything enclosed within the braces is emphasised.

As such, if you want to print the symbols { and }, you can't just type the plain symbol. Instead you must "escape" it. The \ is the escape character and it basically tells LaTeX to pay attention. emph is just a word like any other to LaTeX. But \emph is a command to LaTeX. What escaping a { does is cause LaTeX to print a { character.

So to get {2,4,6,8...}, all you need to do is type:

$\{2, 4, 6, 8\dots\}$

To get {x|x is positive and even}, you can do:

$\{x \mid x \text{ is positive and even}\}$

As raised in the comments - this relies on the amsmath package. Instead, you can do:

$\{x \mid x \mbox{ is positive and even}\}$

For most purposes, this has the same effect, but there are minor differences:

Difference between various methods for producing text in math mode

For really nice spacing, follow Hagen von Eitzen's advice and use:

$\{\,x \mid x \text{ is positive and even}\,\}$

As you can see, \, inserts a thin space. There are other commands for inserting space in LaTeX. However, \, is one of only two that can be used outside of math-mode.

But, really, it's important that you gain an understanding of the basic operating principles of LaTeX before you go further and start worrying about how to do specific things; otherwise you're gonna be completely at sea and everything's gonna seem like black magic and nonsense. So I really recommend you have a little look through some basic introductions before diving right in there.

However I learned by doing - I learned by trying it out, I learned by using LaTeX and looking things up whenever I wanted to do something I didn't yet know how to do. That's a very fulfilling, very enjoyable and very practical way to learn - although there are massive holes in my knowledge and, because my approach was very light on theory, I'm only just starting to learn about bad habits I've developed and principles I may have overlooked.

But still, as good as it is to dive in there and get your hands dirty, you have to, have to get a basic idea of what LaTeX is and how it works or you'll probably continue to have a pretty miserable time. It's no wonder you don't like it and everything seems arcane and difficult and a complete pain in the posterior to you!