[Gnash-commit] [patch #9234] Patch for adding automated slash-colon path checks on `GetVariable()` plugin function

[Nutchanon Wetchasit]

URL:
<http://savannah.gnu.org/patch/?9234>

Summary: Patch for adding automated slash-colon path checks
on `GetVariable()` plugin function
Project: Gnash - The GNU Flash player
Submitted by: nachanon
Submitted on: Mon 23 Jan 2017 11:35:56 PM ICT
Category: testsuite
Priority: 5 - Normal
Status: None
Privacy: Public
Assigned to: None
Originator Email:
Open/Closed: Open
Discussion Lock: Any

_______________________________________________________

Details:

See the attached `0006_add-slashcolon-getvariable-test.patch`.

This patch adds automated tests for Gnash's `GetVariable()` plugin function
to Gnash testsuite. These tests check for problem on slash-colon variable
path
resolving, which is one of the issues tracked in bug #42395;
and fixed in patch #8908 (commit 1d80521
<http://git.savannah.gnu.org/cgit/gnash.git/commit/?id=1d80521d4411e3c3959d9d96989ff9ac8fc36290>).

Notes:
* This test is an automated version of `getvariable-slashcolon.zip`
<https://savannah.gnu.org/bugs/?42395#comment25> originally provided in bug
#42395.
* This test uses emulated container approach with external termination, like
the previously-added patch #8835.
* This test is based on SWFC, as it supports sub-MovieClip.
* Like the original version, coverage is not exactly thorough as it only tests
slash-colon notation without dot.
* This test runs solely with SWF version 7, like half of the tests currently
provided in `misc-swfc.all` testsuite
<http://git.savannah.gnu.org/cgit/gnash.git/tree/testsuite/misc-swfc.all?id=2b3bdede0305c4fc3ad21a0a4197330606c9b880>.
* *This test has 1 expected failure*, as described in the first passing run
<https://savannah.gnu.org/bugs/?42395#comment27>.

Gnash: 0.8.11dev (patched against git 2b3bded 21-Jul-2016)
System: Debian GNU/Linux 7.0 Wheezy i386




_______________________________________________________

File Attachments:


-------------------------------------------------------
Date: Mon 23 Jan 2017 11:35:56 PM ICT Name:
0006_add-slashcolon-getvariable-test.patch Size: 13kB By: nachanon
Patch adding automated slash-colon path checks for `GetVariable()` plugin
function
<http://savannah.gnu.org/patch/download.php?file_id=39541>

_______________________________________________________

Reply to this item at:

<http://savannah.gnu.org/patch/?9234>

_______________________________________________
Message sent via/by Savannah
http://savannah.gnu.org/

[Sandro Santilli]

Update of patch #9234 (project gnash):

Status: None => Need Info

_______________________________________________________

Follow-up Comment #1:

I'm trying to run the .swf with the proprietary player to check it's
correctness but failing, did you succeed ? Can you help with that ? I'd expect
traces in output with "flashplayerdebug" but not getting any. Maybe I didn't
build it correctly (make submoviegetvar.swf)

_______________________________________________________

Reply to this item at:

<http://savannah.gnu.org/patch/?9234>

_______________________________________________
Message sent via/by Savannah
http://savannah.gnu.org/

[Nutchanon Wetchasit]

Follow-up Comment #2, patch #9234 (project gnash):

There might be errors in my patch, but that should not prevent the relevant
tests from being run. The SWF by itself should emit three `PASSED` lines
followed by `ENDOFTEST` in Flash's trace output.

My compilation procedure is as follows:

$ git clean -xdf
$ git checkout -f
Your branch is ahead of 'origin/master' by 1 commit.
$ ./autogen.sh
Running libtoolize --force --copy ...
libtoolize: putting auxiliary files in `.'.
libtoolize: copying file `./ltmain.sh'
libtoolize: putting macros in AC_CONFIG_MACRO_DIR, `macros'.
libtoolize: copying file `macros/libtool.m4'
libtoolize: copying file `macros/ltoptions.m4'
libtoolize: copying file `macros/ltsugar.m4'
libtoolize: copying file `macros/ltversion.m4'
libtoolize: copying file `macros/lt~obsolete.m4'
processing .
Running aclocal -I cygnal -I macros ...
Running autoheader...
Running automake --add-missing --copy ...
configure.ac:35: installing `./compile'
configure.ac:25: installing `./config.guess'
configure.ac:25: installing `./config.sub'
Running autoconf ...
$ CC="ccache gcc" CXX="ccache g++" ./configure --enable-media=ffmpeg,gst
--enable-renderer=agg,cairo,opengl --enable-gui=gtk,qt4,sdl,fb,dump
--enable-docbook
checking build system type... i686-pc-linux-gnu
checking host system type... i686-pc-linux-gnu
checking whether the C++ compiler works... yes
checking for C++ compiler default output file name... a.out
checking for suffix of executables...
.
.
(Output omitted)
.
.
Building Cygnal media server disabled.
Extension support disabled.
Enabling security features: localconnection
Enabling statistics collecting for: cache
GUI toolkits supported: fb,qt4,gtk,sdl,dump
Renderers supported: agg cairo opengl
Hardware Acceleration: RawFB
Media handlers: ffmpeg gst
Using sdl for sound handling
DocBook document processing enabled (for "make html" and "make pdf")
Docbook styles sheets in /usr/share/xml/docbook/stylesheet/nwalsh
Using Input Devices, Touchscreen via Tslib for Input
Using double buffering when rendering

$ cd testsuite/misc-swfc.all
$ make submoviegetvar.swf
ccache gcc -E -x c -P -undef -I.
-DMEDIADIR="/home/window/prog/gnash.git/testsuite/media " submoviegetvar.sc >
pp_submoviegetvar.sc
/usr/bin/swfc -o submoviegetvar.swf pp_submoviegetvar.sc
$ file submoviegetvar.swf
submoviegetvar.swf: Macromedia Flash data (compressed), version 7
$


Trace output from Flash Player runnning `submoviegetvar.swf` is as follows:

Warning: Reference to undeclared variable, 'undefined'
PASSED: [submoviegetvar.sc:40]
Warning: Reference to undeclared variable, 'undefined'
PASSED: [submoviegetvar.sc:46]
Warning: Reference to undeclared variable, 'undefined'
PASSED: [submoviegetvar.sc:52]
ENDOFTEST


I have attached my compiled version of SWF as `submoviegetvar.swf`.

Note: These `PASSED` lines are sanity checks. Real tests are done
by the external test runner.

Gnash testsuite: 0.8.11dev (patched against git 2b3bded 21-Jul-2016)
Flash Player: 11.2.202.491 (NPAPI binary)
Browser: Iceweasel 10.0.12 (debian)
System: Debian GNU/Linux 7.0 Wheezy i386


(file #39554)
_______________________________________________________

Additional Item Attachment:

File name: submoviegetvar.swf Size:1 KB


_______________________________________________________

Reply to this item at:

<http://savannah.gnu.org/patch/?9234>

_______________________________________________
Message sent via/by Savannah
http://savannah.gnu.org/

[Nutchanon Wetchasit]

Follow-up Comment #3, patch #9234 (project gnash):

It is important to note that this test checks interaction
_between Gnash and the external container
<https://savannah.gnu.org/bugs/?37223#comment7>_.

So in order to run it against proprietary player,
a HTML-based test runner must be used instead of shell-based one;
similar to other emulated container-based tests like MTASC-based `extcomm.as`
<http://git.savannah.gnu.org/cgit/gnash.git/tree/testsuite/misc-mtasc.all/extcomm.as?id=2b3bdede0305c4fc3ad21a0a4197330606c9b880>
or MakeSWF-based `extgetvariable.as`
<http://git.savannah.gnu.org/cgit/gnash.git/tree/testsuite/misc-ming.all/extgetvariable.as?id=2b3bdede0305c4fc3ad21a0a4197330606c9b880>.

I have attached the HTML testrunner as `submoviegetvar.html`.
(It is a slightly-modified version of the one provided in
`getvariable-slashcolon.zip`)

Note: You might need to add the test directory as Flash player's
trusted location in order to run tests successfully.

Browser output of my test run went like this (using Flash Player):

PASSED: GetVariable call on "movievar" path should return a correct value
XPASSED: GetVariable call on ":movievar" path should return a correct value
PASSED: GetVariable call on "/:movievar" path should return a correct value
PASSED: GetVariable call on "submovie:submovievar" path should return a
correct value
PASSED: GetVariable call on "submovie/:submovievar" path should return a
correct value
PASSED: GetVariable call on "submovie::submovievar" path should return a
correct value
PASSED: GetVariable call on "/submovie:submovievar" path should return a
correct value
PASSED: GetVariable call on "/submovie/:submovievar" path should return a
correct value
PASSED: GetVariable call on "/submovie::submovievar" path should return a
correct value
PASSED: GetVariable call on ":submovie:submovievar" path should return a
correct value
PASSED: GetVariable call on ":submovie/:submovievar" path should return a
correct value
PASSED: GetVariable call on ":submovie::submovievar" path should return a
correct value
PASSED: GetVariable call on "submovie/nestedmovie:nestedmovievar" path should
return a correct value
PASSED: GetVariable call on "submovie/nestedmovie/:nestedmovievar" path should
return a correct value
PASSED: GetVariable call on "submovie/nestedmovie::nestedmovievar" path should
return a correct value
PASSED: GetVariable call on "submovie:nestedmovie:nestedmovievar" path should
return a correct value
PASSED: GetVariable call on "submovie:nestedmovie/:nestedmovievar" path should
return a correct value
PASSED: GetVariable call on "submovie:nestedmovie::nestedmovievar" path should
return a correct value
PASSED: GetVariable call on "/submovie/nestedmovie:nestedmovievar" path should
return a correct value
PASSED: GetVariable call on "/submovie/nestedmovie/:nestedmovievar" path
should return a correct value
PASSED: GetVariable call on "/submovie/nestedmovie::nestedmovievar" path
should return a correct value
PASSED: GetVariable call on "/submovie:nestedmovie:nestedmovievar" path should
return a correct value
PASSED: GetVariable call on "/submovie:nestedmovie/:nestedmovievar" path
should return a correct value
PASSED: GetVariable call on "/submovie:nestedmovie::nestedmovievar" path
should return a correct value
PASSED: GetVariable call on ":submovie/nestedmovie:nestedmovievar" path should
return a correct value
PASSED: GetVariable call on ":submovie/nestedmovie/:nestedmovievar" path
should return a correct value
PASSED: GetVariable call on ":submovie/nestedmovie::nestedmovievar" path
should return a correct value
PASSED: GetVariable call on ":submovie:nestedmovie:nestedmovievar" path should
return a correct value
PASSED: GetVariable call on ":submovie:nestedmovie/:nestedmovievar" path
should return a correct value
PASSED: GetVariable call on ":submovie:nestedmovie::nestedmovievar" path
should return a correct value


Gnash testsuite: 0.8.11dev (patched against git 2b3bded 21-Jul-2016)
Flash Player: 11.2.202.491 (NPAPI binary)
Browser: Iceweasel 10.0.12 (debian)
System: Debian GNU/Linux 7.0 Wheezy i386


(file #39555)
_______________________________________________________

Additional Item Attachment:

File name: submoviegetvar.html Size:5 KB


_______________________________________________________

Reply to this item at:

<http://savannah.gnu.org/patch/?9234>

_______________________________________________
Message sent via/by Savannah
http://savannah.gnu.org/

[Nutchanon Wetchasit]

Follow-up Comment #4, patch #9234 (project gnash):

For the errors I mentioned <https://savannah.gnu.org/patch/?9234#comment2>
about my patch:

* I used DejaGNU's `check()` function incorrectly; it should be `check(true)`
instead of plain `check()`.
* I didn't track `Dejagnu.sc` and `check.sc` dependency of `submoviegetvar.sc`
in `Makefile`.

Updated patch which fixes these issues, is attached as
`0006_add-slashcolon-getvariable-test-v2.patch`.

My compiled SWF from this patch is attached as `submoviegetvar-v2.zip`,
and can be used together with the previously-posted `submoviegetvar.html`
(file #39555); browser-shown test results should be the same.

Browser output of my `submoviegetvar.html` test run with updated patch
went like this (using Flash Player):

PASSED: GetVariable call on "movievar" path should return a correct value
XPASSED: GetVariable call on ":movievar" path should return a correct value
PASSED: GetVariable call on "/:movievar" path should return a correct value
PASSED: GetVariable call on "submovie:submovievar" path should return a
correct value
PASSED: GetVariable call on "submovie/:submovievar" path should return a
correct value
PASSED: GetVariable call on "submovie::submovievar" path should return a
correct value
PASSED: GetVariable call on "/submovie:submovievar" path should return a
correct value
PASSED: GetVariable call on "/submovie/:submovievar" path should return a
correct value
PASSED: GetVariable call on "/submovie::submovievar" path should return a
correct value
PASSED: GetVariable call on ":submovie:submovievar" path should return a
correct value
PASSED: GetVariable call on ":submovie/:submovievar" path should return a
correct value
PASSED: GetVariable call on ":submovie::submovievar" path should return a
correct value
PASSED: GetVariable call on "submovie/nestedmovie:nestedmovievar" path should
return a correct value
PASSED: GetVariable call on "submovie/nestedmovie/:nestedmovievar" path should
return a correct value
PASSED: GetVariable call on "submovie/nestedmovie::nestedmovievar" path should
return a correct value
PASSED: GetVariable call on "submovie:nestedmovie:nestedmovievar" path should
return a correct value
PASSED: GetVariable call on "submovie:nestedmovie/:nestedmovievar" path should
return a correct value
PASSED: GetVariable call on "submovie:nestedmovie::nestedmovievar" path should
return a correct value
PASSED: GetVariable call on "/submovie/nestedmovie:nestedmovievar" path should
return a correct value
PASSED: GetVariable call on "/submovie/nestedmovie/:nestedmovievar" path
should return a correct value
PASSED: GetVariable call on "/submovie/nestedmovie::nestedmovievar" path
should return a correct value
PASSED: GetVariable call on "/submovie:nestedmovie:nestedmovievar" path should
return a correct value
PASSED: GetVariable call on "/submovie:nestedmovie/:nestedmovievar" path
should return a correct value
PASSED: GetVariable call on "/submovie:nestedmovie::nestedmovievar" path
should return a correct value
PASSED: GetVariable call on ":submovie/nestedmovie:nestedmovievar" path should
return a correct value
PASSED: GetVariable call on ":submovie/nestedmovie/:nestedmovievar" path
should return a correct value
PASSED: GetVariable call on ":submovie/nestedmovie::nestedmovievar" path
should return a correct value
PASSED: GetVariable call on ":submovie:nestedmovie:nestedmovievar" path should
return a correct value
PASSED: GetVariable call on ":submovie:nestedmovie/:nestedmovievar" path
should return a correct value
PASSED: GetVariable call on ":submovie:nestedmovie::nestedmovievar" path
should return a correct value


And this is Flash Player trace log from the run:

Warning: Reference to undeclared variable, 'undefined'
PASSED: [submoviegetvar.sc:40]
Warning: Reference to undeclared variable, 'undefined'
PASSED: [submoviegetvar.sc:46]
Warning: Reference to undeclared variable, 'undefined'
PASSED: [submoviegetvar.sc:52]
ENDOFTEST


Gnash testsuite: 0.8.11dev (patched against git 2b3bded 21-Jul-2016)
Flash Player: 11.2.202.491 (NPAPI binary)
Browser: Iceweasel 10.0.12 (debian)
System: Debian GNU/Linux 7.0 Wheezy i386


(file #39556, file #39557)
_______________________________________________________

Additional Item Attachment:

File name: 0006_add-slashcolon-getvariable-test-v2.patch Size:13 KB
File name: submoviegetvar-v2.zip Size:1 KB


_______________________________________________________

Reply to this item at:

<http://savannah.gnu.org/patch/?9234>

_______________________________________________
Message sent via/by Savannah
http://savannah.gnu.org/

[Nutchanon Wetchasit]

Follow-up Comment #5, patch #9234 (project gnash):

Side note (for the record):

Following warnings:

Warning: Reference to undeclared variable, 'undefined'


which are emitted from DejaGNU's `check()` function are SWFC's fault.
SWFC does not understand the Flash 5+'s `undefined` keyword,
so every reference to `undefined` got compiled as an access to
a variable named "undefined" rather than a special reserved value.

SWFC script like this will show the same error at run time:

.flash bbox=320x240 background=white version=7 fps=12
.frame 1
.action:
var x;
x=undefined;
trace("Hello "+x);
.end
.end


This does not affect test results as variable named "undefined" does not
exist,
so reading from it will (coincidentally) give the expected `undefined` value.

SWFC: SWFTools 0.9.2+git20130725-4 (debian)
Flash Player: 11.2.202.491 (standalone)
System: Debian GNU/Linux 7.0 Wheezy i386

_______________________________________________________

Reply to this item at:

<http://savannah.gnu.org/patch/?9234>

_______________________________________________
Message sent via/by Savannah
http://savannah.gnu.org/

[Sandro Santilli]

Follow-up Comment #6, patch #9234 (project gnash):

Thanks, I can confirm things work now.

I think it would be useful to include the .html file in the sourcecode, for
others to independently check the patch is correct, should I add the one you
attached already or do you want to provide a clean patch, maybe with
references to the .html from the .sc file ?

_______________________________________________________

Reply to this item at:

<http://savannah.gnu.org/patch/?9234>

_______________________________________________
Message sent via/by Savannah
http://savannah.gnu.org/

[Karla]

Follow-up Comment #7, patch #9234 (project gnash):

I was looking for this post. Thanks guys
http://19216801ip.com/

_______________________________________________________

Reply to this item at:

<http://savannah.gnu.org/patch/?9234>

_______________________________________________
Message sent via/by Savannah
http://savannah.gnu.org/

[Nutchanon Wetchasit]

Follow-up Comment #8, patch #9234 (project gnash):

See the attached `0006_add-slashcolon-getvariable-test-v3.patch`
for an updated version of the patch.

This version contains following changes:
* There is now a note about proprietary player testing in
`submoviegetvar.sc`.
* HTML test runner is now included.
** HTML test runner is now counting total tests run.
** HTML test runner is now showing proper `ERROR:` lines about non-loading SWF
or disabled JavaScript.
** HTML test runner now contains comments briefly describing test procedure
and troubleshooting instruction.

Note:
HTML test runner does not count tests run from SWF's side,
as doing so requires multiple changes in `Dejagnu.sc`;
but in this case, there are no real tests done inside the SWF
(apart from sanity checks), so they could be safely ignored.

Gnash: 0.8.11dev (patched against git f9db2f2 26-Jan-2017)
Flash Player: 11.2.202.491 (NPAPI binary)
Browser: Iceweasel 10.0.12 (debian)
System: Debian GNU/Linux 7.0 Wheezy i386


(file #39569)
_______________________________________________________

Additional Item Attachment:

File name: 0006_add-slashcolon-getvariable-test-v3.patch Size:20 KB


_______________________________________________________

Reply to this item at:

<http://savannah.gnu.org/patch/?9234>

_______________________________________________
Message sent via/by Savannah
http://savannah.gnu.org/

[Sandro Santilli]

Update of patch #9234 (project gnash):

Status: Need Info => Ready For Test
Assigned to: None => strk

_______________________________________________________

Follow-up Comment #9:

Thanks, 0006_add-slashcolon-getvariable-test-v3.patch pushed as commit
36028f40cc8fee39d411e13284f2d7334e84b9cc after amending to drop trailing
blanks (git diff --check; git show --check)

Please close after final testing

_______________________________________________________

Reply to this item at:

<http://savannah.gnu.org/patch/?9234>

_______________________________________________
Message sent via/by Savannah
http://savannah.gnu.org/

[Nutchanon Wetchasit]

Follow-up Comment #10, patch #9234 (project gnash):

As of Gnash 0.8.11dev git 36028f4
<http://git.savannah.gnu.org/cgit/gnash.git/commit/?id=36028f40cc8fee39d411e13284f2d7334e84b9cc>,
the newly-added GetVariable() test works as expected. This entry could now be
marked as _Done_. (I don't have a privilege to do this on my own, though)

Test runs are summarized as follows:
* Gnash, out of the box:
** submoviegetvartest-runner: 34 PASSED, 1 XFAILED
** submoviegetvar.html: 30 PASSED, 1 XFAILED
* Gnash, with reverted commit 1d80521
<http://git.savannah.gnu.org/cgit/gnash.git/commit/?id=1d80521d4411e3c3959d9d96989ff9ac8fc36290>:
** submoviegetvartest-runner: 6 PASSED, 1 XFAILED, 28 FAILED
** submoviegetvar.html: 2 PASSED, 1 XFAILED, 28 FAILED
* Flash Player:
** submoviegetvar.html: 30 PASSED, 1 XPASSED

(These figures treats "There should be NN tests run" the same way as other
checks)

Gnash: 0.8.11dev (git 36028f4 28-Jan-2016 07:09:30 GMT)
Ming: 0.4.4-1.1 (debian)
Adobe Flash Player: 11.2.203.491 (NPAPI binary)
Browser: Iceweasel 10.0.12 (debian)
System: Debian GNU/Linux 7.0 Wheezy i386

P.S. Side note: `make check` on latest Gnash version
<http://git.savannah.gnu.org/cgit/gnash.git/commit/?id=ff92e0ae816440b81ecb46215717a376a77491be>
didn't complete due to an unrelated compilation failure involving libming; I
will file a separate report about it later.


_______________________________________________________

Reply to this item at:

<http://savannah.gnu.org/patch/?9234>

_______________________________________________
Message sent via/by Savannah
http://savannah.gnu.org/

[Nutchanon Wetchasit]

Follow-up Comment #11, patch #9234 (project gnash):

The unrelated `make check` error mentioned in comment 10
<https://savannah.gnu.org/patch/?9234#comment10> is now reported as bug
#50174.


_______________________________________________________

Reply to this item at:

<http://savannah.gnu.org/patch/?9234>

_______________________________________________
Message sent via/by Savannah
http://savannah.gnu.org/

[Sandro Santilli]

Update of patch #9234 (project gnash):

Status: Ready For Test => Done
Open/Closed: Open => Closed

_______________________________________________________

Follow-up Comment #12:

Thank you. I agree it was good to open bug #50174

_______________________________________________________

Reply to this item at:

<http://savannah.gnu.org/patch/?9234>

_______________________________________________
Message sent via/by Savannah
http://savannah.gnu.org/