WEBVTT
Kind: captions
Language: en

00:00:01.040 --> 00:00:04.720
Hello everyone! My name is Calle Börstell.

00:00:06.880 --> 00:00:12.320
I sign my name like this and I work here&nbsp;
at the University of Bergen in Norway.

00:00:14.960 --> 00:00:21.760
Today, I will present a demo of my R&nbsp;
library that's called "signglossR".

00:00:27.120 --> 00:00:30.720
So, the first thing we need&nbsp;
is to get another library&nbsp;&nbsp;

00:00:30.720 --> 00:00:36.960
that's connected to my library – so my library&nbsp;
follows the framework of that other library.

00:00:40.160 --> 00:00:43.840
And that's "tidyverse".

00:00:46.000 --> 00:00:51.440
So if you haven't already downloaded signglossR&nbsp;
you need to get it from my Github page.

00:00:55.280 --> 00:01:00.240
So like that, and when you finish,&nbsp;
uh getting the library we need to&nbsp;&nbsp;

00:01:00.240 --> 00:01:05.680
activate it by calling the library.&nbsp;
So that's it, then we're ready to go.

00:01:07.200 --> 00:01:14.160
So one thing that's possible to do with&nbsp;
signglossR is to work with some web-based&nbsp;&nbsp;

00:01:14.160 --> 00:01:19.120
dictionaries, two of them, the Swedish&nbsp;
Sign Language dictionary and ASL Signbank.

00:01:22.480 --> 00:01:26.000
So one thing that's possible&nbsp;
for example is to get images&nbsp;&nbsp;

00:01:26.000 --> 00:01:30.000
from these libraries online&nbsp;
so the dictionary websites.

00:01:35.200 --> 00:01:44.800
So like that. So 1241, that's the number... that's&nbsp;
the ID number from the dictionary and then we get&nbsp;&nbsp;

00:01:44.800 --> 00:01:51.360
that image from the online dictionary...&nbsp;
so we can see it's in the folder here...

00:01:52.720 --> 00:02:00.480
So that... it just... got those two pictures&nbsp;
from the website and combined them into one&nbsp;&nbsp;

00:02:00.480 --> 00:02:07.840
image so the two side-by-side images like this.

00:02:11.840 --> 00:02:16.160
So, it's also possible to do the&nbsp;
same with Signbank if you want to use&nbsp;&nbsp;

00:02:16.160 --> 00:02:23.280
the ASL Signbank you need to&nbsp;
add also the acronym for ASL.

00:02:30.000 --> 00:02:35.840
So there we can see that it got&nbsp;
that sign... the image of that sign.

00:02:38.480 --> 00:02:49.840
It's also possible with videos&nbsp;
so then we can use this function.

00:03:14.000 --> 00:03:17.840
So that means it's easy to&nbsp;
access those dictionaries online.

00:03:21.280 --> 00:03:29.200
So, um... but for example if you&nbsp;
have those two images sometimes&nbsp;&nbsp;

00:03:29.200 --> 00:03:45.840
you want actually both of those&nbsp;
images combined in an overlay.

00:03:49.200 --> 00:03:52.240
So there you can see that automatically&nbsp;&nbsp;

00:03:52.880 --> 00:03:57.040
very quickly it just combined both of&nbsp;
those images into an overlay image...

00:03:57.040 --> 00:04:02.480
... so the first one is a little bit weaker and&nbsp;
the second one is sort of strong, so you can see&nbsp;&nbsp;

00:04:02.480 --> 00:04:13.840
so you can see sort of the movement of the&nbsp;
sign so you see the movement in the sign.

00:04:16.800 --> 00:04:20.800
So with ASL Signbank it's&nbsp;
also possible to add a gloss.

00:04:23.520 --> 00:04:27.840
So with the image.

00:04:29.040 --> 00:04:36.880
So automatically it got the gloss and together&nbsp;
with the image so this looks a little bit&nbsp;&nbsp;

00:04:36.880 --> 00:04:41.840
weird with the big text on top of signer but&nbsp;
it's possible to tweak this a little bit.

00:04:44.800 --> 00:04:51.840
So size change it to 30 and the&nbsp;
gravity argument here is where&nbsp;&nbsp;

00:04:51.840 --> 00:04:55.840
it sort of anchored onto the frame or the image.

00:04:57.760 --> 00:05:00.720
So where the the gloss or&nbsp;
the word actually shows up.

00:05:07.760 --> 00:05:09.840
So that's better!

00:05:12.960 --> 00:05:18.240
So with Swedish Sign... the Swedish Sign Language&nbsp;
dictionary it's also possible to get the GIFs&nbsp;&nbsp;

00:05:18.240 --> 00:05:23.840
because they have GIFs in the dictionary so you&nbsp;
can automatically just get the GIF from there.

00:05:45.440 --> 00:05:52.400
But if there isn't a GIF in the dictionary for&nbsp;
example in Signbank ASL Signbank they don't&nbsp;&nbsp;

00:05:52.400 --> 00:06:05.840
have it so you can still actually you can get the&nbsp;
video and convert it into a GIF with signglossR.

00:06:16.240 --> 00:06:19.440
So in the first part here we're&nbsp;
getting the video and afterwards&nbsp;&nbsp;

00:06:19.440 --> 00:06:33.840
we're actually converting it into a GIF.

00:06:35.600 --> 00:06:41.440
So if you see those uh functions uh they're&nbsp;
combining with the ID numbers from the dictionary&nbsp;&nbsp;

00:06:41.440 --> 00:06:46.000
so you sort of need to know you need to search&nbsp;
through the ID number but what if you don't&nbsp;&nbsp;

00:06:46.000 --> 00:06:52.880
know the ID number of a specific sign well&nbsp;
it's possible to translate the ID numbers to&nbsp;&nbsp;

00:06:52.880 --> 00:07:01.840
a gloss or the other way around from&nbsp;
the gloss back to the ID number.

00:07:07.200 --> 00:07:13.920
For example, this one the number 9 ID number&nbsp;
in the Swedish Sign Language dictionary&nbsp;&nbsp;

00:07:14.640 --> 00:07:29.840
that corresponds to the gloss YNGRE 'younger'.

00:07:32.960 --> 00:07:39.120
So you can see here it it&nbsp;
translates from the ID to the um&nbsp;&nbsp;

00:07:39.120 --> 00:07:47.600
gloss in the ASL Signbank and the reverse as well&nbsp;
so it's easy to use the gloss or the ID number&nbsp;&nbsp;

00:07:47.600 --> 00:07:53.040
um or convert from the gloss to ID&nbsp;
number and then get the sign from there.

00:07:55.760 --> 00:07:58.640
It's the same it got the sign for TOMATO.

00:08:03.200 --> 00:08:10.640
So if, for example, the dictionary has many&nbsp;
signs for a sequential – or maybe it's the same&nbsp;&nbsp;

00:08:10.640 --> 00:08:25.840
also it's possible if you have your own files&nbsp;
in a sequential way – you can combine them.

00:08:29.360 --> 00:08:39.840
So here I have the... my own&nbsp;
images... on my computer.

00:08:50.720 --> 00:09:07.920
And it makes this composite&nbsp;
combined image – one image.

00:09:07.920 --> 00:09:10.880
It's also possible to make an overlay.

00:09:21.520 --> 00:09:26.960
So maybe if you have images of the people and&nbsp;
maybe you don't want to show those people in&nbsp;&nbsp;

00:09:26.960 --> 00:09:35.840
the images maybe you want to hide them or hide&nbsp;
their faces somehow so censor them somehow.

00:09:52.240 --> 00:09:55.120
You can see here automatically&nbsp;
very quickly it just&nbsp;&nbsp;

00:09:56.160 --> 00:10:03.840
added sort of this censoring&nbsp;
blur in front of the face.

00:10:07.040 --> 00:10:17.840
So um if you use ELAN for example it's&nbsp;
possible to make images from the ELAN file.

00:10:19.440 --> 00:10:24.880
So this is just a short video that&nbsp;
I've already finished annotating um&nbsp;&nbsp;

00:10:24.880 --> 00:10:31.840
so there's one segmentation here and four&nbsp;
signs uh segmented into different cells.

00:10:38.640 --> 00:10:43.120
So here I'm just adding the different sort of&nbsp;
arguments here I'm saying what file it is you&nbsp;&nbsp;

00:10:43.120 --> 00:10:59.840
don't file what video file it is where where the&nbsp;
segmentation tier is and where the gloss tier is.

00:11:05.040 --> 00:11:07.680
So auto- it automatically made this combination&nbsp;&nbsp;

00:11:08.400 --> 00:11:14.320
of images so but here you can see that&nbsp;
the gloss here is it's a little bit small.

00:11:19.840 --> 00:11:22.080
So it's possible to tweak it a little bit so&nbsp;&nbsp;

00:11:23.200 --> 00:11:27.840
maybe I want to to reduce the&nbsp;
size of the images a little bit.

00:11:43.120 --> 00:11:50.320
And it's also possible then to combine&nbsp;
with um censoring for instance so it&nbsp;&nbsp;

00:11:50.320 --> 00:11:56.960
um gets the sequential signs um combines&nbsp;
them and sends it to the censoring function&nbsp;&nbsp;

00:11:56.960 --> 00:12:00.640
so each of the images will&nbsp;
then have the faces censored.

00:12:07.120 --> 00:12:09.840
Like that!

00:12:23.520 --> 00:12:29.520
So it's also possible to make video&nbsp;
examples or clips from videos.

00:12:36.720 --> 00:12:37.840
So that's very fast.

00:12:49.760 --> 00:12:53.920
So we're making a video example from&nbsp;
that that's a little bit slower...&nbsp;&nbsp;

00:13:00.800 --> 00:13:06.160
so the speed equals 0.3&nbsp;
means that it's at 30% speed.

00:13:13.440 --> 00:13:19.840
Uh the last thing is also that you can for example&nbsp;
read ELAN files the data from the ELAN files.

00:13:31.600 --> 00:13:38.720
So you can see here you get a list or this data&nbsp;
frame of all these signs that they have in there.

00:13:41.760 --> 00:13:44.320
Or you can read also Openpose...&nbsp;&nbsp;

00:13:55.120 --> 00:14:01.840
so that's a folder that has two different files,

00:14:01.840 --> 00:14:08.080
– and then there are two rows here is&nbsp;
there two two files so two frames uh but&nbsp;&nbsp;

00:14:08.080 --> 00:14:13.440
it's also possible then to get instead&nbsp;
of getting all the the key points here&nbsp;&nbsp;

00:14:13.440 --> 00:14:18.320
in single lines per frame you can get them&nbsp;
separated so you can get them in long format...

00:14:23.960 --> 00:14:24.960
–&nbsp;&nbsp;

00:14:24.960 --> 00:14:30.320
so then you see all the key points they have&nbsp;
each their own sort of row in the data frame.

00:14:35.360 --> 00:14:41.680
Okay great so if you want to know&nbsp;
more about signglossR you can&nbsp;&nbsp;

00:14:42.240 --> 00:14:44.800
go to my the the website of it...

00:14:46.040 --> 00:14:52.480
– so there you can read through and get some more&nbsp;
sort of in-depth or some examples, for example.

00:14:56.960 --> 00:14:59.280
Great, thanks for watching!

