The two tools are different to the point that they are not really interchangeable for most use cases, so it only barely makes sense to have a 'vs' comparison between them. (It would make more sense to compare pkexec vs sudo, or su vs runuser, or pkexec vs run0.)

Code-wise, runuser is practically a copy of util-linux's su, only slightly adapted to be more suitable for use in scripts that generally run as root but need to call some specific command as a different user. (For example, runuser uses a different set of PAM configurations than su, allowing it to be exempt from things like systemd-logind which are often desired for interactive su but would be extremely counter-productive for scripted su.)

The most visible difference between your two commands is that pkexec is usually installed "setuid root" (just like sudo or su), meaning it can function when run from a regular user account – but runuser is usually not installed that way, and in fact specifically refuses to work when run by a non-root user.

This essentially places runuser opposite pkexec, as the latter is meant almost exclusively to raise privileges to root, whereas runuser is only able to lower them from root.

In a similar way, pkexec is designed almost exclusively for interactive desktop use (with PolicyKit prompting the user for admin authentication) while runuser is designed for batch use by root (which would be exempt from such authentication) and therefore doesn't even have the ability to prompt for that.

The only overlap between them is that it is possible for root to call pkexec --user XYZ which will switch to that user without doing the PolicyKit checks (although still contacting polkitd), in which case it has almost no advantages over runuser.

How to configure pkexec to avoid getting errors when run GUI applications?

I found two possible ways:

1. As you can see, using the following:

   pkexec env DISPLAY=$DISPLAY XAUTHORITY=$XAUTHORITY gedit
   

   will not get you any error. And this is normal because man pkexec is very clear in this matter:

          [...] pkexec will not allow you to run X11 applications
          as another user since the $DISPLAY and $XAUTHORITY environment
          variables are not set.[...]
   

   As result you can create an (permanent) alias (this is the simpliest way):

   alias pkexec='pkexec env DISPLAY=$DISPLAY XAUTHORITY=$XAUTHORITY'
   

2. Or, (again) as man pkexec says:

          [...] These two variables will be retained if the
          org.freedesktop.policykit.exec.allow_gui annotation on an action is set
          to a nonempty value; this is discouraged, though, and should only be
          used for legacy programs.[...]
   

   you can create a new policy file in /usr/share/polkit-1/actions named com.ubuntu.pkexec.gedit.policy with the following xml code inside where the most important thing is to set org.freedesktop.policykit.exec.allow_gui to a nonempty value:

   <?xml version="1.0" encoding="UTF-8"?>
   <!DOCTYPE policyconfig PUBLIC
     "-//freedesktop//DTD PolicyKit Policy Configuration 1.0//EN"
     "http://www.freedesktop.org/standards/PolicyKit/1/policyconfig.dtd">
   <policyconfig>
   
     <action id="com.ubuntu.pkexec.gedit">
       <message gettext-domain="gparted">Authentication is required to run gedit</message>
       <icon_name>gedit</icon_name>
       <defaults>
         <allow_any>auth_admin</allow_any>
         <allow_inactive>auth_admin</allow_inactive>
         <allow_active>auth_admin</allow_active>
       </defaults>
       <annotate key="org.freedesktop.policykit.exec.path">/usr/bin/gedit</annotate>
       <annotate key="org.freedesktop.policykit.exec.allow_gui">true</annotate>
     </action>
   
   </policyconfig>
   

How to tell it to not ask for a password after the first time applying it to a command?

For these three setting tags: allow_any, allow_inactive and allow_active from the policy file, the following options are available:

• no: The user is not authorized to carry out the action. There is therefore no need for authentication.
• yes: The user is authorized to carry out the action without any authentication.
• auth_self: Authentication is required but the user need not be an administrative user.
• auth_admin: Authentication as an administrative user is require.
• auth_self_keep: The same as auth_self but, like sudo, the authorization lasts a few minutes.
• auth_admin_keep: The same as auth_admin but, like sudo, the authorization lasts a few minutes.

     ^{Source: Polkit - Structure - Actions}

So, if you use auth_admin_keep option (or, as applicable, auth_self_keep), pkexec will not ask for a password again for some time (by default this time is set to 5 minutes as I checked). The disadvantage here is that this thing is applicable only for one - the same - command / application and valid for all users (unless if it is overruled in later configuration).

Where to save the configuration file if not yet existing?

Configuration files or polkit definitions can be divided into two kinds:

• Actions are defined in XML .policy files located in /usr/share/polkit-1/actions. Each action has a set of default permissions attached to it (e.g. you need to identify as an administrator to use the GParted action). The defaults can be overruled but editing the actions files is NOT the correct way. The name of this policy file should have this format:

  com.ubuntu.pkexec.app_name.policy

• Authorization rules are defined in JavaScript .rules files. They are found in two places: 3rd party packages can use /usr/share/polkit-1/rules.d (though few if any do) and /etc/polkit-1/rules.d is for local configuration. The .rules files designate a subset of users, refer to one (or more) of the actions specified in the actions files and determine with what restrictions these actions can be taken by that/those user(s). As an example, a rules file could overrule the default requirement for all users to authenticate as an admin when using GParted, determining that some specific user doesn't need to. Or isn't allowed to use GParted at all.

     ^{Source: Polkit - Structure}

Is there a GUI application to configure pkexec usage?

From what I know, until now (18.01.2014) doesn't exist something like this. If in the future I will find something, I will not forget to update this answer too.

It can be done by adding custom actions to policykit. If you want to run gedit as root with pkexec you have to create new file /usr/share/polkit-1/actions/org.freedesktop.policykit.gedit.policy for example:

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE policyconfig PUBLIC
 "-//freedesktop//DTD PolicyKit Policy Configuration 1.0//EN"
 "http://www.freedesktop.org/standards/PolicyKit/1/policyconfig.dtd">
<policyconfig>
    <action id="org.freedesktop.policykit.pkexec.gedit">
    <description>Run gedit program</description>
    <message>Authentication is required to run the gedit</message>
    <icon_name>accessories-text-editor</icon_name>
    <defaults>
        <allow_any>auth_admin</allow_any>
        <allow_inactive>auth_admin</allow_inactive>
        <allow_active>auth_admin</allow_active>
    </defaults>
    <annotate key="org.freedesktop.policykit.exec.path">/usr/bin/gedit</annotate>
    <annotate key="org.freedesktop.policykit.exec.allow_gui">true</annotate>
    </action>
</policyconfig>

Finally pkexec gedit should work as expected.

Visit manpage or Reference Manual which explains it with EXAMPLE like:-

$ man pkexec | grep -i ^Example -A 60
EXAMPLE
       To specify what kind of authorization is needed to execute the program /usr/bin/pk-example-frobnicate as
       another user, simply write an action definition file like this

           <?xml version="1.0" encoding="UTF-8"?>
           <!DOCTYPE policyconfig PUBLIC
            "-//freedesktop//DTD PolicyKit Policy Configuration 1.0//EN"
            "http://www.freedesktop.org/standards/PolicyKit/1/policyconfig.dtd">
           <policyconfig>

             <vendor>Examples for the PolicyKit Project</vendor>
             <vendor_url>http://hal.freedesktop.org/docs/PolicyKit/</vendor_url>

             <action id="org.freedesktop.policykit.example.pkexec.run-frobnicate">
               <description>Run the PolicyKit example program Frobnicate</description>
               <description xml:lang="da">Kør PolicyKit eksemplet Frobnicate</description>
               <message>Authentication is required to run the PolicyKit example program Frobnicate (user=$(user), program=$(program), command_line=$(command_line))</message>
               <message xml:lang="da">Autorisering er påkrævet for at afvikle PolicyKit eksemplet Frobnicate (user=$(user), program=$(program), command_line=$(command_line))</message>
               <icon_name>audio-x-generic</icon_name>
               <defaults>
                 <allow_any>no</allow_any>
                 <allow_inactive>no</allow_inactive>
                 <allow_active>auth_self_keep</allow_active>
               </defaults>
               <annotate key="org.freedesktop.policykit.exec.path">/usr/bin/pk-example-frobnicate</annotate>
             </action>

           </policyconfig>

       and drop it in the /usr/share/polkit-1/actions directory under a suitable name (e.g. matching the namespace of
       the action). Note that in addition to specifying the program, the authentication message, description, icon
       and defaults can be specified. Note that occurences of the strings $(user), $(program) and $(command_line) in
       the message will be replaced with respectively the user (of the form "Real Name (username)" or just "username"
       if there is no real name for the username), the binary to execute (a fully-qualified path, e.g.
       "/usr/bin/pk-example-frobnicate") and the command-line, e.g. "pk-example-frobnicate foo bar". For example, for
       the action defined above, the following authentication dialog will be shown:

           [IMAGE][2]

               +----------------------------------------------------------+
               |                     Authenticate                     [X] |
               +----------------------------------------------------------+
               |                                                          |
               |  [Icon]  Authentication is required to run the PolicyKit |
               |          example program Frobnicate                      |
               |                                                          |
               |          An application is attempting to perform an      |
               |          action that requires privileges. Authentication |
               |          is required to perform this action.             |
               |                                                          |
               |          Password: [__________________________________]  |
               |                                                          |
               | [V] Details:                                             |
               |  Command: /usr/bin/pk-example-frobnicate                 |
               |  Run As:  Super User (root)                              |
               |  Action:  org.fd.pk.example.pkexec.run-frobnicate        |
               |  Vendor:  Examples for the PolicyKit Project             |
               |                                                          |
               |                                  [Cancel] [Authenticate] |
               +----------------------------------------------------------+