Wishlist/BigPageWidget

From Openmoko

Revision as of 17:13, 18 March 2010 by Marko Knöbl (Talk | contribs)

(diff) ← Older revision | Latest revision (diff) | Newer revision → (diff)
Jump to: navigation, search
Caveat: While I am not an experienced Linux or GTK programmer, I have a great enthusiasm for and interest in mobile interfaces. This specific idea is based on how much of a joy Picsel Viewer is to use on both the Motorola EZX Platform and Palm OS.
Wishes warning! This article or section documents one or more OpenMoko Wish List items, the features described here may or may not be implemented in the future.

Contents

Introduction

The BigPage Widget is a (proposed, not started) component that any Openmoko application could call upon to handle the display, zooming, scrolling and interaction with pages larger than the screen in a very intuitive and useful way. In the proposed implementation, the BigPage widget would allow the application to create an offscreen 'canvas' of any size that it could render whatever information it desired to. The BigPage widget (placed somewhere on the UI of the application) would then handle the display of the page or part of the page, depending on zoom level.

To simplify viewing, gestures or accelerometer control could be used to zoom, change page, scroll and reposition documents. Kinetic scrolling adds significant 'realism' and intuitiveness to the experience of viewing a large page on a small screen.

It should have a 'text reading mode' that is activated after selecting (double taping?) an are of text. In this mode the text would be reflowed, not scaled, so the user only had to scroll on one direction to read the text.

Interfaces similar to the one proposed here can be as integral to the success of Opera Mini, Picsel Viewer and Mobile Safari. This proposal aims to make that kind of functionality available to all programmers!

high quality text scaling, along with the high resolution of the screen, could allow pages to be read even while heavily scaled.


Why?

The strongest reasons I believe implementing such a widget makes sense are:

  • Scaling of larger pages to with the ability to quickly zoom in and pan in order to read detailed sections is a very natural way to interat with documents on a small screen. It is similar to the way we work with real paper, so intuitive (looking at an overview of a page and looking closer at the details we might read).
  • Fast zooming and scaling are superior to reformatting webpages or documents that were designed to be viewed at conventional screen sizes - it makes the device more useful.
  • Given the likelihood that this method will be adopted by at least some of the applications that make up the Openmoko platform it makes sense to have consistency in the fashion (UI, gestures) in which this kind of interaction is done (especially because to master gestures can take some time)
  • Such a system could be fairly intensive on memory so having a single widget rather than many different implementations of the same idea will reduce memory useage/"dupliction of functionality in memory"
  • It is a desirable feature that many application developers may like to include but be unable to because of lack of experience in writing the necessary graphics code. Making it re-usable and easy to invoke allows cool applications to be written fast, bringing more people to the platform as users and developers
  • It's cool!

Further Explanation

A bigger area to render to

One of the key ideas in BigPage is that an application can render a document at any size it wants. The application developer need not worry about reformating, scaling or anything else. This allows developers to provide users with unfettered rendering of real documents, designed for viewing on a big screen. In business speak, we're making a small device big...

The clearest way I have found to express what is happening is this: The small screen on the mobile device is being used as a window to a larger page. It's as if I've put a big black page with a hole in the centre on my desk and I can easily move around what's underneath so I can see it through the hole. A picture helps here

A page gets rendered to the BigPage widget and initially BigPage displays a scaled version of the whole page (it hasn't reformatted the page, just made it smaller)

http://mailforwho.googlepages.com/bgpage3.jpg

The user chooses to zoom in on one section to start reading, this is done easily, with a pen or finger gesture

http://mailforwho.googlepages.com/BigPage1.jpg http://mailforwho.googlepages.com/bigpage2.jpg

Now imagine moving the page underneath the phone so different bits of it are visible: Because the whole page, or at least the bit on the screen AND the bits close to the edge of the screen, are already rendered panning is fast, it doesn't take any time to redraw the image.

Navigiating around a BigPage

Panning is most intuitively done by tapping and dragging. Picsel implements kinetic scrolling and it is a very natrual and real feeling way for navigation to happen

Links can be clicked on

Zooming could be done by having the AUX-button switch between pan and zoom. In zoom mode simple up/down could change zoom-level.

Gestures

The following gestures are proposed

  • Zoom in or out: A half spiral followed by a drag up or down. Note that Picsel uses a double-tap-drag up or down, but often the first tap gets mistaken for scrolling. Also, the user can choose the centre of the zoom by the centre of the scroll. Alternatively, a tap-and-hold-still-for-a-moment-and-then-drag could be used

http://mailforwho.googlepages.com/zoomgestures.png

  • Zoom around a box: Would use the same gestures as the zoom up and down but after the small spiral the a drag left/right would cause a box to be drawn from the centre of the spiral
  • Page Flick A scroll from the centre of the bottom of the page to the edge (left edge goes backwards, right edge goes forwards)
  • Zoom to see overview: A full spiral:http://mailforwho.googlepages.com/zoomspiral.png
  • Zoom to read This gesture would only apply when zoomed out all the way, it would be a tap on the area the user wanted to read. After the single tap the area would be zoomed in on and the zoom level could be determined by a balance of text size verus an aim to be able to see all of the 'section' tapped upon.

Text

Areas of text, unlike a normal page, benefit greatly from being reflowed/reformatted. Thus, the BigPage widget should detect blocks of text - when these are 'selected' (which would 'feel' similar to clicking on a hyperlink) the mode of the bigpage UI could change from 'scaled display mode' to 'reflowed reading mode'

Reflowed reading mode should have an auto-scroll feature and should be easy to exit to return to 'scaled' mode

User Interface

All the zooming, page turning and scaling options should be available from a UI that could be overlaid on the screen, permanently present at the top or bottom of the display are of the widget.

How might applications use it?

This is very speculative, as no code exists yet, however, It should be possible for any application to include a 'BigPage' widget somewhere on it's UI and that the application can easily render something to that page.

Also, certain properties of the page should be configurable

  • Initial zoom level
  • Whether or not to present a UI to move around
  • How to handle links being clicked (BigPage, surely, must 'know' what a link is in some sense, and the application must complete the handling of the user clicking on a link, which BigPage will notify the app about)

Acceleration

For phones with graphics accelerators it would be good to use them to do better rendering, zooming, etc of the pages.

Personal tools
Caveat: While I am not an experienced Linux or GTK programmer, I have a great enthusiasm for and interest in mobile interfaces. This specific idea is based on how much of a joy Picsel Viewer is to use on both the Motorola EZX Platform and Palm OS.
Wishes warning! This article or section documents one or more OpenMoko Wish List items, the features described here may or may not be implemented in the future.

Introduction

The BigPage Widget is a (proposed, not started) component that any Openmoko application could call upon to handle the display, zooming, scrolling and interaction with pages larger than the screen in a very intuitive and useful way. In the proposed implementation, the BigPage widget would allow the application to create an offscreen 'canvas' of any size that it could render whatever information it desired to. The BigPage widget (placed somewhere on the UI of the application) would then handle the display of the page or part of the page, depending on zoom level.

To simplify viewing, gestures or accelerometer control could be used to zoom, change page, scroll and reposition documents. Kinetic scrolling adds significant 'realism' and intuitiveness to the experience of viewing a large page on a small screen.

It should have a 'text reading mode' that is activated after selecting (double taping?) an are of text. In this mode the text would be reflowed, not scaled, so the user only had to scroll on one direction to read the text.

Interfaces similar to the one proposed here can be as integral to the success of Opera Mini, Picsel Viewer and Mobile Safari. This proposal aims to make that kind of functionality available to all programmers!

high quality text scaling, along with the high resolution of the screen, could allow pages to be read even while heavily scaled.


Why?

The strongest reasons I believe implementing such a widget makes sense are:

  • Scaling of larger pages to with the ability to quickly zoom in and pan in order to read detailed sections is a very natural way to interat with documents on a small screen. It is similar to the way we work with real paper, so intuitive (looking at an overview of a page and looking closer at the details we might read).
  • Fast zooming and scaling are superior to reformatting webpages or documents that were designed to be viewed at conventional screen sizes - it makes the device more useful.
  • Given the likelihood that this method will be adopted by at least some of the applications that make up the Openmoko platform it makes sense to have consistency in the fashion (UI, gestures) in which this kind of interaction is done (especially because to master gestures can take some time)
  • Such a system could be fairly intensive on memory so having a single widget rather than many different implementations of the same idea will reduce memory useage/"dupliction of functionality in memory"
  • It is a desirable feature that many application developers may like to include but be unable to because of lack of experience in writing the necessary graphics code. Making it re-usable and easy to invoke allows cool applications to be written fast, bringing more people to the platform as users and developers
  • It's cool!

Further Explanation

A bigger area to render to

One of the key ideas in BigPage is that an application can render a document at any size it wants. The application developer need not worry about reformating, scaling or anything else. This allows developers to provide users with unfettered rendering of real documents, designed for viewing on a big screen. In business speak, we're making a small device big...

The clearest way I have found to express what is happening is this: The small screen on the mobile device is being used as a window to a larger page. It's as if I've put a big black page with a hole in the centre on my desk and I can easily move around what's underneath so I can see it through the hole. A picture helps here

A page gets rendered to the BigPage widget and initially BigPage displays a scaled version of the whole page (it hasn't reformatted the page, just made it smaller)

http://mailforwho.googlepages.com/bgpage3.jpg

The user chooses to zoom in on one section to start reading, this is done easily, with a pen or finger gesture

http://mailforwho.googlepages.com/BigPage1.jpg http://mailforwho.googlepages.com/bigpage2.jpg

Now imagine moving the page underneath the phone so different bits of it are visible: Because the whole page, or at least the bit on the screen AND the bits close to the edge of the screen, are already rendered panning is fast, it doesn't take any time to redraw the image.

Navigiating around a BigPage

Panning is most intuitively done by tapping and dragging. Picsel implements kinetic scrolling and it is a very natrual and real feeling way for navigation to happen

Links can be clicked on

Zooming could be done by having the AUX-button switch between pan and zoom. In zoom mode simple up/down could change zoom-level.

Gestures

The following gestures are proposed

  • Zoom in or out: A half spiral followed by a drag up or down. Note that Picsel uses a double-tap-drag up or down, but often the first tap gets mistaken for scrolling. Also, the user can choose the centre of the zoom by the centre of the scroll. Alternatively, a tap-and-hold-still-for-a-moment-and-then-drag could be used

http://mailforwho.googlepages.com/zoomgestures.png

  • Zoom around a box: Would use the same gestures as the zoom up and down but after the small spiral the a drag left/right would cause a box to be drawn from the centre of the spiral
  • Page Flick A scroll from the centre of the bottom of the page to the edge (left edge goes backwards, right edge goes forwards)
  • Zoom to see overview: A full spiral:http://mailforwho.googlepages.com/zoomspiral.png
  • Zoom to read This gesture would only apply when zoomed out all the way, it would be a tap on the area the user wanted to read. After the single tap the area would be zoomed in on and the zoom level could be determined by a balance of text size verus an aim to be able to see all of the 'section' tapped upon.

Text

Areas of text, unlike a normal page, benefit greatly from being reflowed/reformatted. Thus, the BigPage widget should detect blocks of text - when these are 'selected' (which would 'feel' similar to clicking on a hyperlink) the mode of the bigpage UI could change from 'scaled display mode' to 'reflowed reading mode'

Reflowed reading mode should have an auto-scroll feature and should be easy to exit to return to 'scaled' mode

User Interface

All the zooming, page turning and scaling options should be available from a UI that could be overlaid on the screen, permanently present at the top or bottom of the display are of the widget.

How might applications use it?

This is very speculative, as no code exists yet, however, It should be possible for any application to include a 'BigPage' widget somewhere on it's UI and that the application can easily render something to that page.

Also, certain properties of the page should be configurable

  • Initial zoom level
  • Whether or not to present a UI to move around
  • How to handle links being clicked (BigPage, surely, must 'know' what a link is in some sense, and the application must complete the handling of the user clicking on a link, which BigPage will notify the app about)

Acceleration

For phones with graphics accelerators it would be good to use them to do better rendering, zooming, etc of the pages.