[RFC] Un-trapping mouse while running fullscreen (SDL limitation/design-flaw)

  Thread Rating:
  • 1 Votes - 5 Average
  • 1
  • 2
  • 3
  • 4
  • 5
Post Reply
bdallen Offline
Junior Member
Posts: 39
Joined: Jun 2008
Reputation: 0
Question  [RFC] Un-trapping mouse while running fullscreen (SDL limitation/design-flaw)
Post: #1
This has come up multiple times on the user forums. If you dual-head using separate X screens, and run XBMC fullscreen on one of the displays, the mouse is trapped and you can't do work on the other screen. This is the way the SDL_FULLSCREEN video mode works, so there does not seem to be an easy fix. switchscreen and wmctrl have been suggested as solutions, but switchscreen works poorly and I can't get wcmtrl to work at all - it just puts my windowed XBMC back into fullscreen with mouse grabbing.

In the hope of finding a solution I've been messing around with a simple SDL app. I think that having the ability to map an input command (mouse, lirc, keyboard, etc) to SDL_WM_ToggleFullScreen would go a long way towards alleviating this problem. Still kludgey, but better than the other solutions I've found.

The idea is that you toggle out of fullscreen, which should allow you to move the mouse to the other screen and do stuff, then when done move the mouse back to focus on the xbmc window and toggle back in. This actually seems to happen when I use switchscreen, but it's not reliable, and is actually unwanted in that case. Also I'm not sure if ToggleFullScreen will still work during video playback.

The downside is that if you want to watch video and work on the other monitor at the same time, you will have the window frame junk. I have never actually wanted to use it this way though - typically I will pause a video and then look up something on imdb or check email.

I will try to work up a patch implementing this. However I am not familiar with the XBMC codebase - I would be very grateful for any advice, or if someone with more experience wants implement it themselves.

Here is the demo app:

Code:
#include <SDL/SDL.h>
#include <stdio.h>
#include <stdlib.h>

int main(int argc, char *argv[])
{
    SDL_Surface *screen;
    if(SDL_Init(SDL_INIT_VIDEO) != 0)
    {
        printf("SDL Init failed: %s", SDL_GetError());
        return 1;
    }

    atexit(SDL_Quit);

    screen = SDL_SetVideoMode(1600, 1200, 16, SDL_FULLSCREEN);
    if(screen == NULL)
    {
        printf("Setting video mode failed: %s", SDL_GetError());
        return 1;
    }

    SDL_Event event;
    int quitFlag = 0;

    while(quitFlag == 0)
    {
        while(SDL_PollEvent(&event))
        {
            if(event.type == SDL_KEYDOWN)
            {
                if (event.key.keysym.sym == SDLK_q
                    || event.key.keysym.sym == SDLK_ESCAPE)
                {
                    quitFlag = 1;
                }
                else if (event.key.keysym.sym == SDLK_f)
                {
                    int success = SDL_WM_ToggleFullScreen(screen);
                    printf("Called WM_ToggleFullScreen, returned %d\n",
                            success);
                }
            }
        }
    }

}
find quote
Gamester17 Offline
Team-XBMC Forum Moderator
Posts: 10,523
Joined: Sep 2003
Reputation: 9
Location: Sweden
Question  Are the same limitations is Simple DirectMedia Layer's latest SVN?
Post: #2
Does SDL 1.3 (SDL latest SVN) have the same limitation?
http://libsdl.org/svn.php

Confused

Always read the XBMC online-manual, FAQ and search the forum before posting.
Do not e-mail XBMC-Team members directly asking for support. Read/follow the forum rules.
For troubleshooting and bug reporting please make sure you read this first.
find quote
bdallen Offline
Junior Member
Posts: 39
Joined: Jun 2008
Reputation: 0
Post: #3
Gamester17 Wrote:Does SDL 1.3 (SDL latest SVN) have the same limitation?
http://libsdl.org/svn.php

Confused
That is a very good question. Based on my attempts below, some bugs need to be fixed in SDL 1.3 before we can answer that question, at least in terms of a test program. However I will post to the SDL mailing list or IRC to see if I can get an answer.

I compiled my fullscreen test app against the latest SVN (as of Sept 19 around 2:00pm CST US), and I get an error: "Setting video mode failed: No video mode large enough for 1600x1200". When I run the tesvidinfo program (in the test directory of the SDL distribution), I get this output:

Code:
Built-in video drivers: x11, dummy
Video driver: x11
Number of displays: 2
Display 0:
  Current mode: 1600x1200@0Hz, 32 bits-per-pixel
      Red Mask = 0x00ff0000
      Green Mask = 0x0000ff00
      Blue Mask = 0x000000ff
No available fullscreen video modes
Display 1:
  Current mode: 1920x1080@0Hz, 32 bits-per-pixel
      Red Mask = 0x00ff0000
      Green Mask = 0x0000ff00
      Blue Mask = 0x000000ff
No available fullscreen video modes

So it would appear that SDL 1.3 is still very much a work in progress.

It looks like the SDL_SetVideoMode is no longer the standard way to initialize video. Many of the tests still use the old API, but the 'CommonInit' function in test/common.c looks to be updated. This function is called by testgl2, testsprite2, and testwm2. I couldn't find documentation on the new API, but looking at the new SDL_video.h is quite informative. The old API functions (like SDL_SetVideoMode) are now in SDL_compat.h.
find quote
CapnBry Offline
Fan
Posts: 406
Joined: Oct 2004
Reputation: 0
Location: Tampa, FL USA
Post: #4
Gamester17 Wrote:Does SDL 1.3 (SDL latest SVN) have the same limitation?
http://libsdl.org/svn.php
Confused
No, because there is no SDL_SetVideoMode any more [Image: emot-v.gif]

To be more informative, the mouse is captured when in SDL fullscreen by design and there is no way to turn it off without unfocusing the windows. 1.3 fullscreen windows also capture the mouse and you can ungrab it, however when focus leaves a fullscreen SDL window it is iconified. I'm pretty sure this is going to defeat the purpose of why people want the mouse to leave the capture rect, having XBMC minimize when you try to do something else.

Solution? Don't use fullscreen mode because that's what fullscreen mode does; getting it to not do that would be like trying to get a leopard to change its spots. What you want is a borderless maximized window. SDL 1.2 sucks in that it doesn't allow you to reposition the window with an SDL call, you have to use native calls to do so.

Good news is 1.3 allows you to create the window SDL_WINDOW_MAXIMIZED | SDL_WINDOW_BORDERLESS with input ingrabbed but if XBMC is relying on SDL to do the mode change too, that will have to change, as mode switching is now independent of window creation.
find quote
bdallen Offline
Junior Member
Posts: 39
Joined: Jun 2008
Reputation: 0
Post: #5
CapnBry Wrote:1.3 fullscreen windows also capture the mouse and you can ungrab it, however when focus leaves a fullscreen SDL window it is iconified. I'm pretty sure this is going to defeat the purpose of why people want the mouse to leave the capture rect, having XBMC minimize when you try to do something else.
Thanks for the reply - it's great to hear from someone who actually knows about SDL! Actually I would not mind if XBMC minimized, since I typically pause the video when I'm doing something else anyway. I think the work and watch scenario is less common, but it would be nice to address both.

CapnBry Wrote:Solution? Don't use fullscreen mode because that's what fullscreen mode does;
Is there a performance hit when running (SDL_WINDOW_MAXIMIZED | SDL_WINDOW_BORDERLESS) vs fullscreen?
find quote
bdallen Offline
Junior Member
Posts: 39
Joined: Jun 2008
Reputation: 0
Post: #6
This patch adds a ToggleFullScreen function to CGraphicContext when HAS_SDL is set. I also modified CXBApplicationEx to call g_graphicContext.ToggleFullScreen() whenever the "f" key is pressed. I realize this is not the proper place (and conflicts with fast-forward). However I had trouble figuring out how to add a command that can be mapped to an LIRC or keyboard event via config files, and wanted to test quickly.

With this patch, I consider the problem "solved", at least well enough for my personal use. First I set XBMC to full resolution, and then (after restarting) it runs in FULLSCREEN with mouse grabbing. Then I can hit "f" to get my mouse back and do work on the other screen, return to the XBMC window and hit "f" again to return to FULLSCREEN. This work even while playing a video, although "f" is also mapped to fast-forward Frown

However as mentioned by CapnBry, if we set XBMC to run maximized but without window decorations, we get "fullscreen" without mouse grab. This can be done in KDE by right clicking on the menu bar and choosing Advanced->No Border (for some reason Fullscreen is greyed out), then using Alt drag to center the borderless window. Having the toggle fullscreen patch allowed me to test this with a full resolution window. Also I'm curious if this incurs a performance hit - it was not noticeable for a ripped DVD, but I have yet to test it with an HD video.

Another question is how to set no-border+maximized programatically (without SDL 1.3/2), and have it work with different window managers. I think wmctrl is designed to do it from the command line, but it doesn't seem to work well with KDE and XBMC beta1. I plan on experimenting with it a bit more, and taking a look at the source code.

Meanwhile I would like to know the proper place to put input-mappable commands. I would really like to get this patch included, because (a) it solves the problem well enough for me and (b) it makes it easier to experiment with other more robust solutions.

Code:
Index: xbmc/XBApplicationEx.cpp
===================================================================
--- xbmc/XBApplicationEx.cpp    (revision 15639)
+++ xbmc/XBApplicationEx.cpp    (working copy)
@@ -399,6 +399,11 @@
       break;
#endif
     case SDL_KEYDOWN:
+      if (event.key.keysym.sym == SDLK_f)
+      {
+        int success = g_graphicsContext.ToggleFullScreen();
+        //printf("Called WM_ToggleFullScreen, returned %d\n", success);
+      }
       g_Keyboard.Update(event);
       // don't handle any more messages in the queue until we've handled keydown,
       // if a keyup is in the queue it will reset the keypress before it is handled.
Index: guilib/GraphicContext.cpp
===================================================================
--- guilib/GraphicContext.cpp   (revision 15639)
+++ guilib/GraphicContext.cpp   (working copy)
@@ -921,6 +921,13 @@
   return g_settings.m_ResInfo[iRes].fPixelRatio;
}

+#ifdef HAS_SDL
+int CGraphicContext::ToggleFullScreen()
+{
+  return SDL_WM_ToggleFullScreen(m_screenSurface->SDL());
+}
+#endif
+
void CGraphicContext::Clear()
{
#ifndef HAS_SDL
Index: guilib/GraphicContext.h
===================================================================
--- guilib/GraphicContext.h     (revision 15639)
+++ guilib/GraphicContext.h     (working copy)
@@ -137,6 +137,7 @@
   void SetD3DParameters(D3DPRESENT_PARAMETERS *p3dParams);
   int GetBackbufferCount() const { return (m_pd3dParams)?m_pd3dParams->BackBufferCount:0; }
#else
+  int ToggleFullScreen();
   inline void setScreenSurface(Surface::CSurface* surface) XBMC_FORCE_INLINE { m_screenSurface = surface; }
   inline Surface::CSurface* getScreenSurface() XBMC_FORCE_INLINE { return m_screenSurface; }
#endif
find quote
Gamester17 Offline
Team-XBMC Forum Moderator
Posts: 10,523
Joined: Sep 2003
Reputation: 9
Location: Sweden
Thumbs Up  Thank you very much!
Post: #7
Great! Have you submitted this patch upstream to libsdl as well?
http://libsdl.org

PS! Please put all patches to XBMC on trac http://trac.xbmc.org

http://wiki.xbmc.org/?title=HOW-TO_submit_a_patch

Lets all hope that this solves the root cause Big Grin

Always read the XBMC online-manual, FAQ and search the forum before posting.
Do not e-mail XBMC-Team members directly asking for support. Read/follow the forum rules.
For troubleshooting and bug reporting please make sure you read this first.
find quote
althekiller Offline
Team-XBMC Developer
Posts: 4,933
Joined: May 2004
Reputation: 12
Post: #8
Maybe I'm missing something here but, what does this do in addition to what the '\' key does already?
find quote
bdallen Offline
Junior Member
Posts: 39
Joined: Jun 2008
Reputation: 0
Post: #9
althekiller Wrote:Maybe I'm missing something here but, what does this do in addition to what the '\' key does already?
Shocked um actually it seems to do almost exactly the same thing. I feel foolish for not looking in the Keymap.xml - I must have really wanted to dig around in the code. Please ignore the above patch.

I'm going to mention this in the user thread. I'm curious to see how many users actually want to continue watching in XBMC while working in another monitor. Maybe it's not worth making a maximized+no-border display mode, at least not until it's easy with a unified API like SDL 1.3/2.
find quote
theuni Offline
Team-XBMC Communication Manager
Posts: 1,107
Joined: Oct 2007
Reputation: 2
Location: Atlanta, Ga, USA
Post: #10
Would it be possible to force SDL to not grab the kb/mouse at all? Or is that essentially the same problem?

I think that lots of people that are using xbmc on 2nd screen/monitor in fullscreen may be controlling it with a remote (myself included).

Maybe not having access to kb/mouse in fullscreen mode will be sufficient for us, as long as we don't lose it on the other monitor?
find quote
bdallen Offline
Junior Member
Posts: 39
Joined: Jun 2008
Reputation: 0
Post: #11
theuni Wrote:Would it be possible to force SDL to not grab the kb/mouse at all? Or is that essentially the same problem?
It doesn't look like this is possible using the SDL API. However it might be possible to run an x11 app without permission to access the keyboard in mouse - it sound like another ugly hack though.

Quote:I think that lots of people that are using xbmc on 2nd screen/monitor in fullscreen may be controlling it with a remote (myself included).
I also use a remote. However when I want to work on my non-XBMC screen I use the keyboard, so having "\" ungrab the mouse works fine. Do you want XBMC to be fullscreen AND have control of the other monitor at the same time? I really think the windows manager fullscreen mode is the best way to get that behavior.
find quote
bdallen Offline
Junior Member
Posts: 39
Joined: Jun 2008
Reputation: 0
Post: #12
bdallen Wrote:Maybe it's not worth making a maximized+no-border display mode, at least not until it's easy with a unified API like SDL 1.3/2.
After poking around a bit more this actually looks easy, even without SDL 1.3.

We can get the x11 Display and Window from here:

http://www.libsdl.org/cgi/docwiki.cgi/SDL_SysWMInfo

and then adapt this function from wmctrl to set the window to WM fullscreen (as opposed to SDL fullscreen):

Code:
static int window_state (Display *disp, Window win, char *arg) {/*{{{*/
    unsigned long action;
    Atom prop1 = 0;
    Atom prop2 = 0;
    char *p1, *p2;
    const char *argerr = "The -b option expects a list of comma separated parameters: \"(remove|add|toggle),<PROP1>[,<PROP2>]\"\n";

    if (!arg || strlen(arg) == 0) {
        fputs(argerr, stderr);
        return EXIT_FAILURE;
    }

    if ((p1 = strchr(arg, ','))) {
        gchar *tmp_prop1, *tmp1;
        
        *p1 = '\0';

        /* action */
        if (strcmp(arg, "remove") == 0) {
            action = _NET_WM_STATE_REMOVE;
        }
        else if (strcmp(arg, "add") == 0) {
            action = _NET_WM_STATE_ADD;
        }
        else if (strcmp(arg, "toggle") == 0) {
            action = _NET_WM_STATE_TOGGLE;
        }
        else {
            fputs("Invalid action. Use either remove, add or toggle.\n", stderr);
            return EXIT_FAILURE;
        }
        p1++;

        /* the second property */
        if ((p2 = strchr(p1, ','))) {
            gchar *tmp_prop2, *tmp2;
            *p2 = '\0';
            p2++;
            if (strlen(p2) == 0) {
                fputs("Invalid zero length property.\n", stderr);
                return EXIT_FAILURE;
            }
            tmp_prop2 = g_strdup_printf("_NET_WM_STATE_%s", tmp2 = g_ascii_strup(p2, -1));
            p_verbose("State 2: %s\n", tmp_prop2);
            prop2 = XInternAtom(disp, tmp_prop2, False);
            g_free(tmp2);
            g_free(tmp_prop2);
        }

        /* the first property */
        if (strlen(p1) == 0) {
            fputs("Invalid zero length property.\n", stderr);
            return EXIT_FAILURE;
        }
        tmp_prop1 = g_strdup_printf("_NET_WM_STATE_%s", tmp1 = g_ascii_strup(p1, -1));
        p_verbose("State 1: %s\n", tmp_prop1);
        prop1 = XInternAtom(disp, tmp_prop1, False);
        g_free(tmp1);
        g_free(tmp_prop1);

        
        return client_msg(disp, win, "_NET_WM_STATE",
            action, (unsigned long)prop1, (unsigned long)prop2, 0, 0);
    }
    else {
        fputs(argerr, stderr);
        return EXIT_FAILURE;
    }
}/*}}}*/

I'll try to work up a patch later today.
find quote
bdallen Offline
Junior Member
Posts: 39
Joined: Jun 2008
Reputation: 0
Post: #13
I have updated my demo app so that the "g" key toggles the fullscreen property on the window. There are some odd interactions between SDL's fullscreen mode and the windowed fullscreen mode, but I think they can be avoided by not setting the windowed fullscreen mode unless SDL fullscreen is disabled.

Just to be clear, what I am referring to as "windowed fullscreen" is a maximized window without a border.

See this page to download the tarball.

Some questions:

1) Are the XBMC devs interested in adding a windowed fullscreen mode?

2) If so, how should this be enabled? Through a separate boolean option on Settings->Appearance->Screen? Or maybe just a config file option, since most people don't need this? Should it replace the SDL fullscreen when enabled, so that "\" would toggle between maximized+no border and bordered? It might be nice to toggle all three, especially if maximized+no border doesn't perform as well as SDL fullscreen.

3) Should we try to set an "Always on top" property also? When I enter windowed fullscreen with my demo app, the KDE panel still displays over my app until I click. Perhaps just focusing the window would solve this.

4) Does this work in Gnome and other window managers? It should, since it uses the same protocol as wmctrl, which Gnome apparently supports.

Also some thoughts about including this in SDL: it sounds like you will be able to set the required window properties using SDL 1.3/2, so I doubt they will be interested in adding it to 1.2.X. Maybe if someone created a really good patch, but I think developer time could be better spent elseware.
find quote
bdallen Offline
Junior Member
Posts: 39
Joined: Jun 2008
Reputation: 0
Post: #14
Gamester17 Wrote:Great! Have you submitted this patch upstream to libsdl as well?
http://libsdl.org

PS! Please put all patches to XBMC on trac http://trac.xbmc.org

http://wiki.xbmc.org/?title=HOW-TO_submit_a_patch

Lets all hope that this solves the root cause Big Grin
Thanks for the links! Once I have a good patch I'll post it on trac. I'm not sure that getting a patch into SDL is feasible or a good idea. The "accepted wisdom" about SDL fullscreen seems to be that the mouse grabbing is part of it's fundamental nature, and trying to change that is silly. I'm not completely satisfied that this is true, but I'm also not qualified to question it at the moment.

What follows is a bunch of poorly informed speculation about the nature of SDL fullscreen. Hopefully someone else will find it interesting, or take the time to substantiate or refute it. Otherwise it was useful for organizing my thoughts on the subject Rolleyes.

I think the reason that SDL fullscreen is a separate video mode is that it's a fundamentally a different way of interacting with the graphics subsystem. In X11 it seems to bypass the window manager. It might be different in OS X and Windows, and unlocking the mouse might be easy on some platforms but not others. The issue is further complicated by the many modes of dual-heading on each platform, many of which seem to be based on how the hardware is used to drive the different heads.

If CapnBry is correct, the proper way to implement "no mouse grab fullscreen" for Linux/X11 is using a windowed video mode with a maximized borderless window, and SDL 1.3 already has API calls to allow this. If SDL fullscreen really is different, it seems like bad API design to have an option for SDL fullscreen with no grabbing. This approach probably works on other platforms also.

Adding the feature to 1.2 seems like wasted effort - I think it would amount to backporting the relevent 1.3 features, which would be a major effort. Just getting it working in X11 for windows managers that support EWMH/NetWM may not be so bad, but there are many other platforms supported by SDL, and it seems unlikely they would accept a new feature that only worked on one platform.

When (if at all?) does XBMC plan to switch to 1.3/2? I get the impression it won't be out for a while. I really think that the best use of developer time is to implement the windowed fullscreen using the native X11 calls, like I've done with my demo app, and then when SDL 2 comes out refactor and use SDL to do the same thing.

The SDL source code (and mailing list) could probably answer these questions more reliably... I should be writing my thesis but I might take a look. Productive procrastination FTW.
find quote
althekiller Offline
Team-XBMC Developer
Posts: 4,933
Joined: May 2004
Reputation: 12
Post: #15
Keep in mind that one of the goals of XBMC is platform neutrality. Something like this would need to be able to be implemented win32 and osx as well.
find quote
Post Reply