it-e-69 Image Compression

Why is image compression so important? Image files, in an uncompressed form, are very large.
And the Internet, especially for people using a 56k dialup modem, can be pretty slow.[1]This
combination could seriously limit one of the Web's most appreciated aspectsits ability to present
images easily.
JPEG (Joint Photographic Experts Group)compression is currently the best way to compress
PHOTOGRAPHIC IMAGES for the web. Other forms of image compression, including GIF and
PNG, are best used for other purposes on the web.
GIF (Graphics Interchange Format) is best used for graphics that have a limited color pallet
and large areas of flat tone, like cartoons or banners. Although it has several remarkable features,
such as transparency and the ability to present animated images, it is not well suited for the
presentation of continuous tone images, such as photographs, due to its limit of 256 colors.
PNG (Portable Network Graphics) is a relatively new format with a lot of potential but, until
all browsers can see images compressed in PNG form, it is not a good idea to use it.
JPEG, or JPG, is an evolving format that is universal in its use as a means of compressing
continuous tone photographs for speedy transmission over the Internet. Photographs compressed
using the JPEG format look good because JPEG supports millions of colors, so you can see the
gradation of tones.
A Bitmap is a simple series of pixels all stacked up. But the same image saved in GIF or
JPEG format uses less bytes to make up the file. How? Compression.
"Compression" is a computer term that represents a variety of mathematical formats used to
compress an image's byte size. Let's say you have an image where the upper right-hand corner
has four pixels all the same color. Why not find a way to make those four pixels into one?[2]That
would cut down the number of bytes by three-fourths, at least in the one corner. That's a
compression factor.
Bitmaps can be compressed to a point. The process is called "run-length encoding." Runs of
pixels that are all the same color are all combined into one pixel. [3]The longer the run of pixels,

the more compression. Bitmaps with little detail or color variance will really compress. Those
with a great deal of detail don't offer much in the way of compression. Bitmaps that use the
run-length encoding can carry either the common ".bmp" extension or ".rle". Another difference
between the two files is that the common Bitmap can accept 16 million different colors per pixel.
Saving the same image in run-length encoding knocks the bits-per-pixel down to 8. That locks
the level of color in at no more than 256.
So, why not create a single pixel when all of the colors are close? You could even lower the
number of colors available so that you would have a better chance of the pixels being close in
color. Good idea. The people at CompuServe felt the same way.
GIF, which stands for "Graphic Interchange Format," was first standardized in 1987 by
CompuServe, although the patent for the algorithm (mathematical formula) used to create GIF
compression actually belongs to Unisys. The first format of GIF used on the Web was called
GIF87a, representing its year and version. It saved images at 8 bits-per-pixel, capping the color
level at 256. That 8-bit level allowed the image to work across multiple server styles, including
CompuServe, TCP/IP.
CompuServe updated the GIF format in 1989 to include animation, transparency, and interlacing.
They called the new format, you guessed it: GIF89a.
There's no discernable difference between a basic (known as non-interlaced) GIF in 87 and
89 formats.

 

1, continuous tone  
连续色调

2, gradation  [ɡrə'deiʃən]
n. (色彩、颜色、次序、音调等的)渐变;分等级;(各种状态、性质等的)分阶段渐变;元音交替

3, discernable  [di'sə:nəbl, -'zə:-]
adj. 可辨别的;可认识的

4, interlaced  [,intə'leist]
a. 交织的,交错的

5, interlacing  [intə(:)'leisiŋ]
n. 交错,隔行;隔行扫描
v. 交错,交织(interlace现在分词)

Continue reading it-e-69 Image Compression

【转+评】淘宝数据魔方技术架构解析

程序员原文:http://www.programmer.com.cn/7578/
读着很像论文体,故此收藏。其中很多概念可以先读读百科hadoop http://baike.baidu.com/view/908354.htmhttp://www.infoq.com/cn/articles/hadoop-intro

淘宝网拥有国内最具商业价值的海量数据。截至当前,每天有超过30亿的店铺、商品浏览记录,10亿在线商品数,上千万的成交、收藏和评价数据。如何从这些数据中挖掘出真正的商业价值,进而帮助淘宝、商家进行企业的数据化运营,帮助消费者进行理性的购物决策,是淘宝数据平台与产品部的使命。

为此,我们进行了一系列数据产品的研发,比如为大家所熟知的量子统计、数据魔方和淘宝指数等。尽管从业务层面来讲,数据产品的研发难度并不高;但在 “海量”的限定下,数据产品的计算、存储和检索难度陡然上升。本文将以数据魔方为例,向大家介绍淘宝在海量数据产品技术架构方面的探索。

淘宝海量数据产品技术架构

数据产品的一个最大特点是数据的非实时写入,正因为如此,我们可以认为,在一定的时间段内,整个系统的数据是只读的。这为我们设计缓存奠定了非常重要的基础。

图1 淘宝海量数据产品技术架构
图1 淘宝海量数据产品技术架构

按照数据的流向来划分,我们把淘宝数据产品的技术架构分为五层(如图1所示),分别是数据源、计算层、存储层、查询层和产品层。位于架构顶端的是我们的数据来源层,这里有淘宝主站的用户、店铺、商品和交易等数据库,还有用户的浏览、搜索等行为日志等。这一系列的数据是数据产品最原始的生命力所在。

在数据源层实时产生的数据,通过淘宝自主研发的数据传输组件DataX、DbSync和Timetunnel准实时地传输到一个有1500个节点的 Hadoop[注:apache开源分布文件系统,主要用途搜索引擎。]集群上,这个集群我们称之为“云梯”,是计算层的主要组成部分。在“云梯”上,我们每天有大约40000个作业对1.5PB的原始数据按照产品需求进行不同的MapReduce[注:Google提出的概念,MapReduce是一种编程模型,用于大规模数据集(大于1TB)的并行运算。概念"Map(映射)"和"Reduce(化简)",和他们的主要思想,都是从函数式编程语言里借来的,还有从矢量编程语言里借来的特性。他极大地方便了编程人员在不会分布式并行编程的情况下,将自己的程序运行在分布式系统上]计算。这一计算过程通常都能在凌晨两点之前完成。相对于前端产品看到的数据,这里的计算结果很可能是一个处于中间状态的结果,这往往是在数据冗余与前端计算之间做了适当平衡的结果。

不得不提的是,一些对实效性要求很高的数据,例如针对搜索词的统计数据,我们希望能尽快推送到数据产品前端。这种需求再采用“云梯”来计算效率将是比较低的,为此我们做了流式数据的实时计算平台,称之为“银河”。“银河”也是一个分布式系统,它接收来自TimeTunnel的实时消息,在内存中做实时计算,并把计算结果在尽可能短的时间内刷新到NoSQL存储设备中,供前端产品调用。

容易理解,“云梯”或者“银河”并不适合直接向产品提供实时的数据查询服务。这是因为,对于“云梯”来说,它的定位只是做离线计算的,无法支持较高的性能和并发需求;而对于“银河”而言,尽管所有的代码都掌握在我们手中,但要完整地将数据接收、实时计算、存储和查询等功能集成在一个分布式系统中,避免不了分层,最终仍然落到了目前的架构上。

为此,我们针对前端产品设计了专门的存储层。在这一层,我们有基于MySQL的分布式关系型数据库集群MyFOX和基于HBase的NoSQL存储集群Prom,在后面的文字中,我将重点介绍这两个集群的实现原理。除此之外,其他第三方的模块也被我们纳入存储层的范畴。

存储层异构模块的增多,对前端产品的使用带来了挑战。为此,我们设计了通用的数据中间层——glider——来屏蔽这个影响。glider以HTTP协议对外提供restful方式的接口。数据产品可以通过一个唯一的URL获取到它想要的数据。

以上是淘宝海量数据产品在技术架构方面的一个概括性的介绍,接下来我将重点从四个方面阐述数据魔方设计上的特点。

关系型数据库仍然是王道

关系型数据库(RDBMS)自20世纪70年代提出以来,在工业生产中得到了广泛的使用。经过三十多年的长足发展,诞生了一批优秀的数据库软件,例如Oracle、MySQL、DB2、Sybase和SQL Server等。

图2 MyFOX中的数据增长曲线
图2 MyFOX中的数据增长曲线

尽管相对于非关系型数据库而言,关系型数据库在分区容忍性(Tolerance to Network Partitions)方面存在劣势,但由于它强大的语义表达能力以及数据之间的关系表达能力,在数据产品中仍然占据着不可替代的作用。

淘宝数据产品选择MySQL的MyISAM引擎作为底层的数据存储引擎。在此基础上,为了应对海量数据,我们设计了分布式MySQL集群的查询代理层——MyFOX,使得分区对前端应用透明。

图3 MyFOX的数据查询过程
图3 MyFOX的数据查询过程

目前,存储在MyFOX中的统计结果数据已经达到10TB,占据着数据魔方总数据量的95%以上,并且正在以每天超过6亿的增量增长着(如图2所示)。这些数据被我们近似均匀地分布到20个MySQL节点上,在查询时,经由MyFOX透明地对外服务(如图3所示)。

图4 MyFOX节点结构
图4 MyFOX节点结构

值得一提的是,在MyFOX现有的20个节点中,并不是所有节点都是“平等”的。一般而言,数据产品的用户更多地只关心“最近几天”的数据,越早的数据,越容易被冷落。为此,出于硬件成本考虑,我们在这20个节点中分出了“热节点”和“冷节点”(如图4所示)。

顾名思义,“热节点”存放最新的、被访问频率较高的数据。对于这部分数据,我们希望能给用户提供尽可能快的查询速度,所以在硬盘方面,我们选择了每分钟15000转的SAS硬盘,按照一个节点两台机器来计算,单位数据的存储成本约为4.5W/TB。相对应地,“冷数据”我们选择了每分钟7500转的 SATA硬盘,单碟上能够存放更多的数据,存储成本约为1.6W/TB。

将冷热数据进行分离的另外一个好处是可以有效降低内存磁盘比。从图4可以看出,“热节点”上单机只有24GB内存,而磁盘装满大约有 1.8TB(300 * 12 * 0.5 / 1024),内存磁盘比约为4:300,远远低于MySQL服务器的一个合理值。内存磁盘比过低导致的后果是,总有一天,即使所有内存用完也存不下数据的索引了——这个时候,大量的查询请求都需要从磁盘中读取索引,效率大打折扣。

NoSQL是SQL的有益补充

在MyFOX出现之后,一切都看起来那么完美,开发人员甚至不会意识到MyFOX的存在,一条不用任何特殊修饰的SQL语句就可以满足需求。这个状态持续了很长一段时间,直到有一天,我们碰到了传统的关系型数据库无法解决的问题——全属性选择器(如图5所示)。

图5 全属性选择器
图5 全属性选择器

这是一个非常典型的例子。为了说明问题,我们仍然以关系型数据库的思路来描述。对于笔记本电脑这个类目,用户某一次查询所选择的过滤条件可能包括 “笔记本尺寸”、“笔记本定位”、“硬盘容量”等一系列属性(字段),并且在每个可能用在过滤条件的属性上,属性值的分布是极不均匀的。在图5中我们可以看到,笔记本电脑的尺寸这一属性有着10个枚举值,而“蓝牙功能”这个属性值是个布尔值,数据的筛选性非常差。

在用户所选择的过滤条件不确定的情况下,解决全属性问题的思路有两个:一个是穷举所有可能的过滤条件组合,在“云梯”上进行预先计算,存入数据库供查询;另一个是存储原始数据,在用户查询时根据过滤条件筛选出相应的记录进行现场计算。很明显,由于过滤条件的排列组合几乎是无法穷举的,第一种方案在现实中是不可取的;而第二种方案中,原始数据存储在什么地方?如果仍然用关系型数据库,那么你打算怎样为这个表建立索引?

这一系列问题把我们引到了“创建定制化的存储、现场计算并提供查询服务的引擎”的思路上来,这就是Prometheus(如图6所示)。

图6 Prom的存储结构
图6 Prom的存储结构

从图6可以看出,我们选择了HBase作为Prom的底层存储引擎。之所以选择HBase,主要是因为它是建立在HDFS之上的,并且对于 MapReduce有良好的编程接口。尽管Prom是一个通用的、解决共性问题的服务框架,但在这里,我们仍然以全属性选择为例,来说明Prom的工作原理。这里的原始数据是前一天在淘宝上的交易明细,在HBase集群中,我们以属性对(属性与属性值的组合)作为row-key进行存储。而row-key 对应的值,我们设计了两个column-family,即存放交易ID列表的index字段和原始交易明细的data字段。在存储的时候,我们有意识地让每个字段中的每一个元素都是定长的,这是为了支持通过偏移量快速地找到相应记录,避免复杂的查找算法和磁盘的大量随机读取请求。

图7 Prom查询过程
图7 Prom查询过程

图7用一个典型的例子描述的Prom在提供查询服务时的工作原理,限于篇幅,这里不做详细描述。值得一提的是,Prom支持的计算并不仅限于求和 SUM运算,统计意义上的常用计算都是支持的。在现场计算方面,我们对Hbase进行了扩展,Prom要求每个节点返回的数据是已经经过“本地计算”的局部最优解,最终的全局最优解只是各个节点返回的局部最优解的一个简单汇总。很显然,这样的设计思路是要充分利用各个节点的并行计算能力,并且避免大量明细数据的网络传输开销。

用中间层隔离前后端

上文提到过,MyFOX和Prom为数据产品的不同需求提供了数据存储和底层查询的解决方案,但随之而来的问题是,各种异构的存储模块给前端产品的使用带来了很大的挑战。并且,前端产品的一个请求所需要的数据往往不可能只从一个模块获取。

举个例子,我们要在数据魔方中看昨天做热销的商品,首先从MyFOX中拿到一个热销排行榜的数据,但这里的“商品”只是一个ID,并没有ID所对应的商品描述、图片等数据。这个时候我们要从淘宝主站提供的接口中去获取这些数据,然后一一对应到热销排行榜中,最终呈现给用户。

图8 glider的技术架构
图8 glider的技术架构

有经验的读者一定可以想到,从本质上来讲,这就是广义上的异构“表”之间的JOIN操作。那么,谁来负责这个事情呢?很容易想到,在存储层与前端产品之间增加一个中间层,它负责各个异构“表”之间的数据JOIN和UNION等计算,并且隔离前端产品和后端存储,提供统一的数据查询服务。这个中间层就是glider(如图8所示)。

缓存是系统化的工程

除了起到隔离前后端以及异构“表”之间的数据整合的作用之外,glider的另外一个不容忽视的作用便是缓存管理。上文提到过,在特定的时间段内,我们认为数据产品中的数据是只读的,这是利用缓存来提高性能的理论基础。

在图8中我们看到,glider中存在两层缓存,分别是基于各个异构“表”(datasource)的二级缓存和整合之后基于独立请求的一级缓存。除此之外,各个异构“表”内部可能还存在自己的缓存机制。细心的读者一定注意到了图3中MyFOX的缓存设计,我们没有选择对汇总计算后的最终结果进行缓存,而是针对每个分片进行缓存,其目的在于提高缓存的命中率,并且降低数据的冗余度。

大量使用缓存的最大问题就是数据一致性问题。如何保证底层数据的变化在尽可能短的时间内体现给最终用户呢?这一定是一个系统化的工程,尤其对于分层较多的系统来说。

图9 缓存控制体系
图9 缓存控制体系

图9向我们展示了数据魔方在缓存控制方面的设计思路。用户的请求中一定是带了缓存控制的“命令”的,这包括URL中的query string,和 HTTP头中的“If-None-Match”信息。并且,这个缓存控制“命令”一定会经过层层传递,最终传递到底层存储的异构“表”模块。各异构“表” 除了返回各自的数据之外,还会返回各自的数据缓存过期时间(ttl),而glider最终输出的过期时间是各个异构“表”过期时间的最小值。这一过期时间也一定是从底层存储层层传递,最终通过HTTP头返回给用户浏览器的。

缓存系统不得不考虑的另一个问题是缓存穿透与失效时的雪崩效应。缓存穿透是指查询一个一定不存在的数据,由于缓存是不命中时被动写的,并且出于容错考虑,如果从存储层查不到数据则不写入缓存,这将导致这个存在的数据每次请求都要到存储层去查询,失去了缓存的意义。

有很多种方法可以有效地解决缓存穿透问题,最常见的则是采用布隆过滤器,将所有可能存在的数据哈希到一个足够大的bitmap中,一个一定不存在的数据会被这个bitmap拦截掉,从而避免了对底层存储系统的查询压力。在数据魔方里,我们采用了一个更为简单粗暴的方法,如果一个查询返回的数据为空(不管是数据不存在,还是系统故障),我们仍然把这个空结果进行缓存,但它的过期时间会很短,最长不超过五分钟。

缓存失效时的雪崩效应对底层系统的冲击非常可怕。遗憾的是,这个问题目前并没有很完美的解决方案。大多数系统设计者考虑用加锁或者队列的方式保证缓存的单线程(进程)写,从而避免失效时大量的并发请求落到底层存储系统上。在数据魔方中,我们设计的缓存过期机制理论上能够将各个客户端的数据失效时间均匀地分布在时间轴上,一定程度上能够避免缓存同时失效带来的雪崩效应。

结束语

正是基于本文所描述的架构特点,数据魔方目前已经能够提供压缩前80TB的数据存储空间,数据中间层glider支持每天4000万的查询请求,平均响应时间在28毫秒(6月1日数据),足以满足未来一段时间内的业务增长需求。

尽管如此,整个系统中仍然存在很多不完善的地方。[这句话真是太熟了]一个典型的例子莫过于各个分层之间使用短连接模式的HTTP协议进行通信。这样的策略直接导致在流量高峰期单机的TCP连接数非常高。所以说,一个良好的架构固然能够在很大程度上降低开发和维护的成本,但它自身一定是随着数据量和流量的变化而不断变化的。我相信,过不了几年,淘宝数据产品的技术架构一定会是另外的样子。

Continue reading 【转+评】淘宝数据魔方技术架构解析

it-e-68 Can Graphics Files Be Enctypted

Of course you can encrypt a graphics file. After all, most encryption algorithms don't care
about the intellectual content of a file. All they chew on is a series of byte values. Therefore,
most any encryption program that works on ordinary text files will work on graphics files as well.
Why would you want to encrypt a graphics file? Mostly to control who can view its contents.
You can invent a proprietary file format and that might slow a file format hack down for, say,
five or ten minutes. You could add a proprietary data compression scheme, possibly a twisted
variation of an already public algorithm. But there are so many people out there with nothing
better to do than hack at unknown data formats that your data would probably be exposed in little
time. But suppose we top off all this effort by encrypting the graphics file itself as we would an
ordinary text file. Would your data then be safe?
Realize that an encrypted graphics file still might not be very secure. For every data
encryption algorithm there exists at least one method of getting around it, although it may take
hundreds of computers and many years to fully employ and execute that method!
For example, one of the more popular methods used to encrypt data is the Vernam or XOR
cipher. This cipher Exclusive ORs the plain-text data with a single, random, fixed-length key.
The longer the key the harder it is to break the cipher. A totally random key the length of your
data is impossible to break. Shorter and less-random keys are easier to break.
XOR is very simple and fast, which is a must for a graphics file translators/viewers that
must decrypt a file on the fly. A problem, however, is that most graphics files contain fixed size
headers which vary only slightly in content from file to file. If you knew the approximate
contents of the header of an encrypted file you could XOR a "decrypted" header with the
encrypted file and possibly produce the key used to encrypt the file. A short key might be very
easily discovered in this way.
If you wish to use a public key/private key encryption method, then storing the public key in
the file format header (usually as a 4-byte field) and only encrypting the image data would be the
way to go. The SMPTE DPX file format supports such an encryption feature.
If you really need to make the contents of a graphics file secure, then I'd suggest not only
using some form of data encryption, but also create an unconventional and proprietary file format
and do not publish its format specification.

cipher ['saifə]

n. 零,暗号

  • v. 计算,做算术

Continue reading it-e-68 Can Graphics Files Be Enctypted

it-e-67 Can Graphics Files Be Infected with a Virus

For most types of graphics file formats currently available the answer is "no". A virus (or
worm, Trojan horse, and so forth) is fundamentally a collection of code (that is, a program) that
contains instructions which are executed by a CPU. Most graphics files, however, contain only
static data and no executable code. The code that reads, writes, and displays graphics data is
found in translation and display programs and not in the graphics files themselves. If reading or
writing a graphics file caused a system malfunction is it most likely the fault of the program
reading the file and not of the graphics file data itself.
With the introduction of multimedia we have seen new formats appear, and modifications to
older formats made, that allow executable instructions to be stored within a file format. These
instructions are used to direct multimedia applications to play sounds or music, prompt the user

for information, or display other graphics and video information. And such multimedia display
programs may perform these functions by interfacing with their environment via an API, or by
direct interaction with the operating system. One might also imagine a truly object-oriented
graphics file as containing the code required to read, write, and display itself.
Once again, any catastrophes that result from using these multimedia application is most
like the result of unfound bugs in the software and not some sinister instructions in the graphics
file data. Such "logic bombs" are typically exorcised through the use of testing using a wide
variety of different image files for test cases.
If you have a virus scanning program that indicates a specific graphics file is infected by
virus, then it is very possible that the file coincidentally contains a byte pattern that the scanning
programming recognizes as a key byte signature identifying a virus. Contact the author (or even
read the documentation!) of the virus scanning program to discuss the probability of the
mis-identification of a clean file as being infected by a virus. Save the graphics file, as the author
will most likely wish to examine it as well.
If you suspect a graphics file to be at the heart of a virus problem you are experiencing, then
also consider the possibility that the graphics file's transport mechanism (floppy disk, tape or
shell archive file, compressed archive file, and so forth) might be the original source of the virus
and not the graphics file itself.

1, catastrophe  [kə'tæstrəfi]
n. 大灾难;大祸;惨败

2, sinister  ['sinistə]
a. 不吉利的,凶恶的,左边的

3, exorcise  ['eksɔ:saiz]
vt. 驱邪;除怪

4, coincidentally  [kəu,insi'dentli]
adv. 巧合地;一致地

Continue reading it-e-67 Can Graphics Files Be Infected with a Virus

it-e-66 Introduction to Digital Image Processing

An image is digitized to convert it to a form that can be stored in a computer's memory or on

some form of storage media such as a hard disk or CD-ROM. This digitization procedure can be
done by a scanner, or by a video camera connected to a frame grabber board in a computer. Once an
image has been digitized, it can be operated upon by various image processing operations.
Image processing operations can be roughly divided into three major categories, Image
Compression, Image Enhancement and Restoration, and Measurement Extraction[提取尺寸]. Image
compression is familiar to most people. [1]It involves reducing the amount of memory needed to
store a digital image.
Image defects which could be caused by the digitization process or by faults in the imaging
set-up (for example, bad lighting) can be corrected using Image Enhancement techniques. Once
the image is in good condition, the Measurement Extraction operations can be used to obtain
useful information from the image.
Some examples of Image Enhancement and Measurement Extraction are given below. The
examples shown all operate on 256 grey-scale images. This means that each pixel in the image is
stored as a number between 0 to 255, where 0 represents a black pixel, 255 represents a white
pixel and values in-between represent shades of grey. These operations can be extended to
operate on colour images.
The examples below represent only a few of the many techniques available for operating on
images. Details about the inner workings of the operations have not been given.
Image Enhancement and Restoration
The image at the left of Figure 1 has been corrupted image
by noise during the digitization process. The "clean"
image at the right of Figure 1 was obtained by applying a
median filter [中值滤波器]to the image.
An image with poor contrast, such as the one at
the left of Figure 2, can be improved by adjusting the
image histogram to produce the image shown at the
right of Figure 2.

 image
The image at the top left of Figure 3 has a corrugated effect due to a fault in the acquisition
process. This can be removed by doing a 2-dimensional Fast-Fourier Transform[快速傅里叶变换] on the image
(top right of Figure 3), removing the bright spots (bottom left of Figure 3), and finally doing an
inverse Fast Fourier Transform to return to the original image without the corrugated background
  (bottom right of Figure 3).

image

An image which has been captured in poor lighting conditions, and shows a continuous
change in the background brightness across the image (top left of Figure 4) can be corrected
using the following procedure. First remove the foreground objects by applying a 25 by 25
greyscale dilation operation (top right of Figure 4). Then subtract the original image from the
background image (bottom left of Figure 4). Finally invert the colors and improve the contrast by
adjusting the image histogram (bottom right of Figure 4).

image

The example below demonstrates how one could go about extracting measurements from an
image. The image at the top left of Figure 5 shows some objects. The aim is to extract information
about the distribution of the sizes (visible areas) of the objects. The first step involves segmenting
the image to separate the objects of interest from the background. This usually involves
thresholding the image, which is done by setting the values of pixels above a certain threshold value
to white, and all the others to black (top right of Figure 5). Because the objects touch, thresholding
at a level which includes the full surface of all the objects does not show separate objects. This
problem is solved by performing a watershed separation on the image (lower left of Figure 5). The
image at the lower right of Figure 5 shows the result of performing a logical AND of the two
images at the left of Figure 5. This shows the effect that the watershed separation has on touching
objects in the original image. Finally, some measurements can be extracted from the image.

image

frame grabber board  帧中继访问设备

 

1, corrugated  ['kɔrəgeitid]
a. 缩成皱纹的,使起波状的

2, dilation  [dai'leiʃən, di-]
n. 扩张,扩大;膨胀;详述

3, subtract  [səb'trækt]
v. 减去,扣掉,减少

4, segmenting  
n. 分段

5, thresholding  
n. 阈值转换法;域值

6, watershed  ['wɔ:təʃed, 'wɔ-]
n. (美)流域;分水岭;集水区;转折点
adj. 标志转折点的

Continue reading it-e-66 Introduction to Digital Image Processing

it-e-65 Convert a Graphics Format

Why change vectors to bitmaps?
Most of the clip art gallery is vector-based and will need to be converted into bitmap formats
(GIF) prior to putting it on the Web.
Why change bitmaps to vectors?
You will need to change vectors to bitmaps to perform tasks from the Drawing toolbar on a
bitmap picture (such as animate parts of a bitmap picture) you will need to convert it into a vector
format. You can then e.g. ungroup it and apply animations to only parts of it.
Which graphic converter to use?
To change your graphics format, you need to use a graphics converter. A popular graphics
editor you can use for this is Paint Shop Pro. Another graphics editor you can use is Adobe
PhotoShop, which is said to be the best one for this kind of conversion.
How to use your graphic converter?
Open your file in the graphics editor chosen: Select File | Open.
Select File | Save As.
Rename your file and choose a new format. For a bitmap to vectors conversion select the
WMF format. For the opposite conversion, select the GIF format if you have PowerPoint 97,
otherwise select JPEG or TIFF.
Unfortunately, some of the quality may be lost in the switch. MS office also provides
graphics converters.

Continue reading it-e-65 Convert a Graphics Format

phprpc初步

官网:

http://phprpc.org/

java只有源代码,不过包含了个make.bat运行后就生成了三个jar包,服务端只需要phprpc.jar,客户端需要phprpc_client.jar,不需要别的依赖包,very nice,那个phprpc_spring.jar的包尚不知是做什么用的。

服务端,自定义接口:

>package kzg.phprpc.hello.api;

Continue reading phprpc初步

it-3-64 Course Description

This course is an introduction to the basic concepts as well as applications of the rapidly
emerging field of digital image processing. It familiarizes the audience with the understanding,
design, and implementation of algorithms in the various subareas of digital image processing
such as image enhancement, image deblurring, image understanding, image security, and image
compression. Over 200 image examples complement the technical descriptions.
Benefits/Learning Objectives
This course will enable you to
explain the fundamental concepts and terminologies employed in digital imaging such as
sampling and aliasing, perceptual quantization; filtering, look-up tables, image histogram,
etc;

explain the various techniques used in image enhancement for contrast manipulation (e.g.,
histogram equalization), sharpening (e.g., unsharp masking) and noise removal (e.g.,
selective averaging, median filtering);
briefly demonstrate the performance of image deblurring algorithms such as inverse filtering
and Wiener filtering by using image examples;
briefly demonstrate the concepts behind digital signatures for image authentication and
invisible watermarking for image copyright protection;
briefly describe the current research topics in image understanding and demonstrate related
algorithm performances using image examples;
explain the basic technologies that serve the existing JPEG and the emerging JPEG2000
standards.
Intended Audience
Scientists, engineers, and managers who need to understand and/or apply the fundamental
concepts and techniques employed in digital image processing. Although no particular background is
needed, some prior knowledge of linear system theory (e.g., Fourier transforms) would be helpful.

1, deblurring  
n. [计] 去模糊
v. 由模糊变清晰;擦掉…的污点(deblur的ing形式)

2, histogram  ['histəugræm]
n. 柱状图
[计算机] 直方图

3, perceptual  [pə'septjuəl]
a. 感性的,知觉的

4, quantization  [,kwɔntai'zeiʃən]
量子化,数字化

5, Fourier  ['furiei]
n. 傅里叶

Continue reading it-3-64 Course Description

Pagination


Total views.

© 2013 - 2019. All rights reserved.

Powered by Hydejack v6.6.1