Philips Win Patent Infringement Suit Against Nintendo
This moggy has not played a computer game since the Atari console system in the early 1980s, and so felt technically and culturally unqualified to write about the Philips and Nintendo litigation which has resulted in the recent judgment from the Patents Court. Fortunately, his younger colleagues Jacob Loftus and Gemma Wooden have come to his aid with this report.
This was a decision by Mr Justice Birss in the Patents Court ([2014] EWHC 1959 (Pat)). Philips claimed that Nintendo’s popular Wii and Wii U consoles infringed three of its patents. The first patent was EP0808484 entitled "Method and apparatus for controlling the movement of a virtual body” and related to a user directed representation of a virtual body in a virtual environment. The second patent, EP1573498, entitled "User interface system based on a pointing device", related to controlling electronic devices using a pointer. The third patent, EP2093650, was a divisional of the 498 patent and related to very similar subject matter. Philips has also brought actions against Nintendo in the US, France and Germany as reported here.
484 patent
The first Philips patent under consideration related to “virtual body modelling” and a user directed representation of a virtual body in a virtual environment. In general terms, the system comprised two data stores where the first data store contained information about a virtual environment and the second data store contained a data store for motions of a virtual body in the virtual environment. The user performing a certain sequence of motions causes a motion detector to send a signal to the system which in turn results in the virtual body performing a sequence of motions which follow the user’s motions.
Nintendo sought revocation of the 484 patent citing four documents against the patent as granted. Philips proposed amending claim 1 such that the virtual body modelling apparatus included an “adaptive mechanism” where the mechanism was “arranged to adapt on the fly to the signals received from the user motion detection means to translate the user's erratic, variable signals into a steady motion”.
Kat readers familiar with the Wii may also be aware of the “Balance Board”. The Balance Board contains pressure sensors which are used to determine the user’s centre of balance. Applying a repetitive action such as jumping or running on the board causes the avatar to perform a similar action on the screen. Sometimes the motion on screen is a distinct motion from that performed by the user but the motion on screen is responsive to, for example, an increase or decrease of speed by the user. In the present case, Philips claimed that the game “Island Cycling”, in which a player’s running motion on the balance board becomes a cycling action of an avatar on screen, infringed claim 1. Birss J considered this from the point of view of claim 1 with the amendments proposed by Philips. The focus was on whether the system translated user’s erratic motions into a steady motion. Philips submitted an experiment modelling erratic motion which demonstrated that when a user took a step on the floor instead of the board did not affect the steady motion of the cyclist on screen. Birss J agreed and that the Wii playing Island Cyclist does perform the necessary function to infringe. It is difficult to disagree with this – the Wii system frequently compensates for users’ erratic motions to aid gameplay.
Obviousness of the 484 patent
One of the three prior art items cited by Nintendo was the use of a device called a Power Pad on the original Nintendo NES console with a game called World Class Track Meet (WCTM) and in particular the ‘100m dash’ mini-game in WCTM.
Similarly to the Wii Balance Board, the Power Pad for the NES relied on pressure sensing pads. Philips contended that WCTM was not a virtual environment within the meaning of claim 1 of 484 and therefore the apparatus was not present. Birss J agreed that WCTM did not constitute a virtual environment as would be understood by the skilled person at the relevant time. Consequently claim 1 was found to be novel over WCTM. However, assessing inventive step, Birss J found that all the claims lacked inventive step over WCTM with the power pad on the NES system, as the skilled person at the priority date, when presented with the power pad and WCTM, would naturally have upgraded WCTM to a virtual environment with the available graphics at that time.
In summary, claims 1 and 5 were found to be infringed by the Wii running the Island Cycling game in combination with the Balance Board. However, all the claims were found to be invalid on grounds of lack of inventive step.
498 and 650 patents
Both patents related to handheld pointing devices used to control electrical apparatus and the IPKat will focus on the 498 patent. The setup is as in Fig. 1 (see right).
The 498 patent described a portable handheld device which could be used to point at and control a device such as a Hi-Fi system. The pointing device could additionally be used in the following way: gestures could also be detected by the system by means of a camera mounted on the pointing device. The motion trajectory of the pointing device could then be decoded by the system and used as another means of controlling devices in the room.
In proceedings Philips put forward amendments to the claims of the 498 patent to deal with an added matter issue, novelty and inventive step issues over the prior art and a typographical error in claim 1. The final form of claim 1 as accepted by the court to be amended during the proceedings was as follows:
Nintendo contended that claim 1 as granted contained added matter extending beyond the application as filed. There were two separate arguments put forward. Firstly, claim 1 as granted referred to the use of “at least one room localisation beacon” whereas the application as filed never disclosed the use of a single beacon. Birss J agreed with this point and claim 1 as granted was found invalid. Nintendo’s second argument related to the use of the beacons. Essentially they argued the function of the beacons in the application was strictly identification improvement and that due to various amendments during prosecution the patent as granted covered other cases such as gestures. This argument was rejected on the grounds that it would be clear to the skilled addressee that the use of beacons to help in determining which part of a room was being pointed to would not be so limited as to the use of the beacons for simple identification improvement and, actually other kinds of uses of the beacons are implicitly disclosed.
It was accepted that claim 1 as granted of the 498 patent lacked novelty over Wacom (Japanese unexamined patent application No. H07-302148 published on 14 November 1995 entitled "Data input device"). As amended above, however, it was held that the patent was novel and inventive over this document, as well as Philips (PCT Application WO 00/60534 published on 12 November 2000 entitled "Remote control for display apparatus") and Sony (Japanese unexamined patent application 2002-81909 published on 22 March 2002 entitled "Position detector, position detection method, and entertainment apparatus").
Nintendo also contended that the proposed amendments to the 498 patent should not be permitted by the court because they would lead to double patenting. Birss J held that a double patenting objection could be taken as a ground for refusing a post-grant amendment to a claim but only in the following circumstances:
(ii) The two patents have the same priority dates and are held by the same applicant (or its successor in title);
(ii) The two claims must have the same scope;
(iii) The two claims must be independent claims;
(iv) If the objection arises in the Patents Court and both patents are before the court, it can be cured by amending either patent;
(v) If the patentee has a legitimate interest in maintaining both claims then the amendment should not be refused, even if the other criteria are met.
In this case, it was held that the amendments to the claims were allowable because, although the two putative independent claims (claim 1 of 498 as set out above, and claim 2 of 650 as dependent on claim 1) overlap, there is a material difference in their scope. Moreover, Birss J held that the patentee had a legitimate interest in maintaining the amended claims of the 498 patent, but not in also making the validating amendment (claim 2 into claim 1) that would needed for the 650 patent.
Infringement
The alleged infringement, according to Philips, occurs when the Wii is running the Wii Tennis game. Wii Tennis is fairly self-explanatory: the user makes gestures similar to tennis motions and the avatar on screen makes a similar, though not exactly identical, motion in a virtual tennis environment.
Nintendo’s case for non-infringement was on two grounds. The first was the fact that the Wii system cannot recognise which part of the room the Wii remote is pointing to. In fact it only recognises when it is pointing at the Wii sensor (a small bar of LEDs placed in the vicinity of the console). In other words, recognition only takes place within a frame of reference in which the LED bar is fixed. As rightly observed by Birss J, this is not sufficient to bring the Wii outside the claim. Second, an argument based on a scenario involving the Wii being able to distinguish a left handed from a right handed person was put forward by Nintendo but this was also dismissed.
There were further issues discussed relating to the prior art documents which led to the partial invalidity of aspects of the claims. In the end, Birss J identified a combination of elements which were valid and infringed by the Wii playing Wii Tennis. The issues were summarised in paragraph 450 of the judgment:
Kat followers who keep an eye on the gaming industry will be aware that Nintendo’s Wii U console has suffered from fairly bad press and poor sales figures since its release, reported here, however since the release of Mario Kart 8 and announcements of other big titles this year at the gaming expo E3, the fortunes of Nintendo looked set to turn around. Consequently, the ruling could prove unfortunate for Nintendo should Philips seek to restrict sales of the Wii U. Nintendo are appealing the decision.
This was a decision by Mr Justice Birss in the Patents Court ([2014] EWHC 1959 (Pat)). Philips claimed that Nintendo’s popular Wii and Wii U consoles infringed three of its patents. The first patent was EP0808484 entitled "Method and apparatus for controlling the movement of a virtual body” and related to a user directed representation of a virtual body in a virtual environment. The second patent, EP1573498, entitled "User interface system based on a pointing device", related to controlling electronic devices using a pointer. The third patent, EP2093650, was a divisional of the 498 patent and related to very similar subject matter. Philips has also brought actions against Nintendo in the US, France and Germany as reported here.
484 patent
The first Philips patent under consideration related to “virtual body modelling” and a user directed representation of a virtual body in a virtual environment. In general terms, the system comprised two data stores where the first data store contained information about a virtual environment and the second data store contained a data store for motions of a virtual body in the virtual environment. The user performing a certain sequence of motions causes a motion detector to send a signal to the system which in turn results in the virtual body performing a sequence of motions which follow the user’s motions.
 |
| Mario Kat. |
Nintendo sought revocation of the 484 patent citing four documents against the patent as granted. Philips proposed amending claim 1 such that the virtual body modelling apparatus included an “adaptive mechanism” where the mechanism was “arranged to adapt on the fly to the signals received from the user motion detection means to translate the user's erratic, variable signals into a steady motion”.
Kat readers familiar with the Wii may also be aware of the “Balance Board”. The Balance Board contains pressure sensors which are used to determine the user’s centre of balance. Applying a repetitive action such as jumping or running on the board causes the avatar to perform a similar action on the screen. Sometimes the motion on screen is a distinct motion from that performed by the user but the motion on screen is responsive to, for example, an increase or decrease of speed by the user. In the present case, Philips claimed that the game “Island Cycling”, in which a player’s running motion on the balance board becomes a cycling action of an avatar on screen, infringed claim 1. Birss J considered this from the point of view of claim 1 with the amendments proposed by Philips. The focus was on whether the system translated user’s erratic motions into a steady motion. Philips submitted an experiment modelling erratic motion which demonstrated that when a user took a step on the floor instead of the board did not affect the steady motion of the cyclist on screen. Birss J agreed and that the Wii playing Island Cyclist does perform the necessary function to infringe. It is difficult to disagree with this – the Wii system frequently compensates for users’ erratic motions to aid gameplay.
Obviousness of the 484 patent
One of the three prior art items cited by Nintendo was the use of a device called a Power Pad on the original Nintendo NES console with a game called World Class Track Meet (WCTM) and in particular the ‘100m dash’ mini-game in WCTM.
Similarly to the Wii Balance Board, the Power Pad for the NES relied on pressure sensing pads. Philips contended that WCTM was not a virtual environment within the meaning of claim 1 of 484 and therefore the apparatus was not present. Birss J agreed that WCTM did not constitute a virtual environment as would be understood by the skilled person at the relevant time. Consequently claim 1 was found to be novel over WCTM. However, assessing inventive step, Birss J found that all the claims lacked inventive step over WCTM with the power pad on the NES system, as the skilled person at the priority date, when presented with the power pad and WCTM, would naturally have upgraded WCTM to a virtual environment with the available graphics at that time.
In summary, claims 1 and 5 were found to be infringed by the Wii running the Island Cycling game in combination with the Balance Board. However, all the claims were found to be invalid on grounds of lack of inventive step.
498 and 650 patents
Both patents related to handheld pointing devices used to control electrical apparatus and the IPKat will focus on the 498 patent. The setup is as in Fig. 1 (see right).The 498 patent described a portable handheld device which could be used to point at and control a device such as a Hi-Fi system. The pointing device could additionally be used in the following way: gestures could also be detected by the system by means of a camera mounted on the pointing device. The motion trajectory of the pointing device could then be decoded by the system and used as another means of controlling devices in the room.
In proceedings Philips put forward amendments to the claims of the 498 patent to deal with an added matter issue, novelty and inventive step issues over the prior art and a typographical error in claim 1. The final form of claim 1 as accepted by the court to be amended during the proceedings was as follows:
User interaction system, comprising:
- an electrical apparatus (110);
- a portable pointing device (101, 300) operable by a user for pointing to a region in space;
- a camera (102) taking a picture;
- motion sensing means, and
- a digital signal processor (120), capable of receiving and processing the picture, and capable of transmitting user interface information (I) derived from the picture to the electrical apparatus (110),
wherein the camera (102) is connected to the pointing device (101, 300) so that in operation it images the region pointed to, the system being characterised in that it further comprises room localization beacons (180, 181, 182), the system being, in a room wherein the pointing device is used, that can emit electromagnetic radiation, for use by the digital signal processor (120) in order to recognise which part of the room the pointing device is pointing; and the digital signal processor (120) is further arranged to recognise to which part of the room the pointing device is pointing and wherein the digital signal processor is arranged to analyse gestures made with the pointing device based upon a motion trajectory (400, 410) of the pointing device.
Nintendo contended that claim 1 as granted contained added matter extending beyond the application as filed. There were two separate arguments put forward. Firstly, claim 1 as granted referred to the use of “at least one room localisation beacon” whereas the application as filed never disclosed the use of a single beacon. Birss J agreed with this point and claim 1 as granted was found invalid. Nintendo’s second argument related to the use of the beacons. Essentially they argued the function of the beacons in the application was strictly identification improvement and that due to various amendments during prosecution the patent as granted covered other cases such as gestures. This argument was rejected on the grounds that it would be clear to the skilled addressee that the use of beacons to help in determining which part of a room was being pointed to would not be so limited as to the use of the beacons for simple identification improvement and, actually other kinds of uses of the beacons are implicitly disclosed.
.png) |
| Example of Wacom device |
Nintendo also contended that the proposed amendments to the 498 patent should not be permitted by the court because they would lead to double patenting. Birss J held that a double patenting objection could be taken as a ground for refusing a post-grant amendment to a claim but only in the following circumstances:
(ii) The two patents have the same priority dates and are held by the same applicant (or its successor in title);
(ii) The two claims must have the same scope;
(iii) The two claims must be independent claims;
(iv) If the objection arises in the Patents Court and both patents are before the court, it can be cured by amending either patent;
(v) If the patentee has a legitimate interest in maintaining both claims then the amendment should not be refused, even if the other criteria are met.
In this case, it was held that the amendments to the claims were allowable because, although the two putative independent claims (claim 1 of 498 as set out above, and claim 2 of 650 as dependent on claim 1) overlap, there is a material difference in their scope. Moreover, Birss J held that the patentee had a legitimate interest in maintaining the amended claims of the 498 patent, but not in also making the validating amendment (claim 2 into claim 1) that would needed for the 650 patent.
Infringement
The alleged infringement, according to Philips, occurs when the Wii is running the Wii Tennis game. Wii Tennis is fairly self-explanatory: the user makes gestures similar to tennis motions and the avatar on screen makes a similar, though not exactly identical, motion in a virtual tennis environment.
 |
| Wii sensor bar |
There were further issues discussed relating to the prior art documents which led to the partial invalidity of aspects of the claims. In the end, Birss J identified a combination of elements which were valid and infringed by the Wii playing Wii Tennis. The issues were summarised in paragraph 450 of the judgment:
"i) The patent application which led to the 498 patent discloses a computer system with a hand held pointing device which has both a camera and a physical motion sensor such as an accelerometer. It is used to give hand waving gesture commands to a fixed unit. The gesture analysis is based on the motion trajectory of the device. The system uses room localisation beacons (plural) but not necessarily three beacons. One application of this combination which is described is to use it for playing games.
ii) No prior art discloses that combination. Wacom has no physical motion sensor and no gesture analysis. The Philips application describes a pointing device with a camera but the idea of using it for hand waving gestures is not disclosed. The device can have a tilt sensor but that was just an activation switch. Sony is a game system with a hand held unit which used a camera but at best it is a near miss. It does not disclose the idea of monitoring the swing. It has no physical motion sensor.
iii) For inventive step, none of the cited prior art leads naturally to the claimed combination and pointers to it are not in the common general knowledge. The common general knowledge did not include a device combining a physical motion sensor with a camera and the reasons advanced by Nintendo for putting those two sensors together in one unit are unconvincing. Also the common general knowledge did not include any game based on analysing hand waving gestures. The gesture examples relied on are two dimensional mouse gestures (like Graffiti or Black & White) or contrived (like the joystick cheat codes).
iv) Both patents were granted with broader claims but those broader claims are invalid in various ways. However a claim to the combination described is not added matter because it is disclosed in the patent application. The Nintendo Wii system set up with Wii Tennis is an example of that combination and therefore infringes the claim."
Kat followers who keep an eye on the gaming industry will be aware that Nintendo’s Wii U console has suffered from fairly bad press and poor sales figures since its release, reported here, however since the release of Mario Kart 8 and announcements of other big titles this year at the gaming expo E3, the fortunes of Nintendo looked set to turn around. Consequently, the ruling could prove unfortunate for Nintendo should Philips seek to restrict sales of the Wii U. Nintendo are appealing the decision.
