Update CodeRay version to 1.0 final (#4264).

git-svn-id: svn+ssh://rubyforge.org/var/svn/redmine/trunk@7618 e93f8b46-1217-0410-a6f0-8f06a7374b81
This commit is contained in:
Etienne Massip committed 2011-10-08 13:34:30 +00:00
1 parent 6ebb5e834c
commit d1efb4f148
115 files changed
+3482 -17507

No files matched your search

+504
View File
@@ -0,0 +1,504 @@
GNU LESSER GENERAL PUBLIC LICENSE
Version 2.1, February 1999
Copyright (C) 1991, 1999 Free Software Foundation, Inc.
51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA
Everyone is permitted to copy and distribute verbatim copies
of this license document, but changing it is not allowed.
[This is the first released version of the Lesser GPL. It also counts
as the successor of the GNU Library Public License, version 2, hence
the version number 2.1.]
Preamble
The licenses for most software are designed to take away your
freedom to share and change it. By contrast, the GNU General Public
Licenses are intended to guarantee your freedom to share and change
free software--to make sure the software is free for all its users.
This license, the Lesser General Public License, applies to some
specially designated software packages--typically libraries--of the
Free Software Foundation and other authors who decide to use it. You
can use it too, but we suggest you first think carefully about whether
this license or the ordinary General Public License is the better
strategy to use in any particular case, based on the explanations below.
When we speak of free software, we are referring to freedom of use,
not price. Our General Public Licenses are designed to make sure that
you have the freedom to distribute copies of free software (and charge
for this service if you wish); that you receive source code or can get
it if you want it; that you can change the software and use pieces of
it in new free programs; and that you are informed that you can do
these things.
To protect your rights, we need to make restrictions that forbid
distributors to deny you these rights or to ask you to surrender these
rights. These restrictions translate to certain responsibilities for
you if you distribute copies of the library or if you modify it.
For example, if you distribute copies of the library, whether gratis
or for a fee, you must give the recipients all the rights that we gave
you. You must make sure that they, too, receive or can get the source
code. If you link other code with the library, you must provide
complete object files to the recipients, so that they can relink them
with the library after making changes to the library and recompiling
it. And you must show them these terms so they know their rights.
We protect your rights with a two-step method: (1) we copyright the
library, and (2) we offer you this license, which gives you legal
permission to copy, distribute and/or modify the library.
To protect each distributor, we want to make it very clear that
there is no warranty for the free library. Also, if the library is
modified by someone else and passed on, the recipients should know
that what they have is not the original version, so that the original
author's reputation will not be affected by problems that might be
introduced by others.
Finally, software patents pose a constant threat to the existence of
any free program. We wish to make sure that a company cannot
effectively restrict the users of a free program by obtaining a
restrictive license from a patent holder. Therefore, we insist that
any patent license obtained for a version of the library must be
consistent with the full freedom of use specified in this license.
Most GNU software, including some libraries, is covered by the
ordinary GNU General Public License. This license, the GNU Lesser
General Public License, applies to certain designated libraries, and
is quite different from the ordinary General Public License. We use
this license for certain libraries in order to permit linking those
libraries into non-free programs.
When a program is linked with a library, whether statically or using
a shared library, the combination of the two is legally speaking a
combined work, a derivative of the original library. The ordinary
General Public License therefore permits such linking only if the
entire combination fits its criteria of freedom. The Lesser General
Public License permits more lax criteria for linking other code with
the library.
We call this license the "Lesser" General Public License because it
does Less to protect the user's freedom than the ordinary General
Public License. It also provides other free software developers Less
of an advantage over competing non-free programs. These disadvantages
are the reason we use the ordinary General Public License for many
libraries. However, the Lesser license provides advantages in certain
special circumstances.
For example, on rare occasions, there may be a special need to
encourage the widest possible use of a certain library, so that it becomes
a de-facto standard. To achieve this, non-free programs must be
allowed to use the library. A more frequent case is that a free
library does the same job as widely used non-free libraries. In this
case, there is little to gain by limiting the free library to free
software only, so we use the Lesser General Public License.
In other cases, permission to use a particular library in non-free
programs enables a greater number of people to use a large body of
free software. For example, permission to use the GNU C Library in
non-free programs enables many more people to use the whole GNU
operating system, as well as its variant, the GNU/Linux operating
system.
Although the Lesser General Public License is Less protective of the
users' freedom, it does ensure that the user of a program that is
linked with the Library has the freedom and the wherewithal to run
that program using a modified version of the Library.
The precise terms and conditions for copying, distribution and
modification follow. Pay close attention to the difference between a
"work based on the library" and a "work that uses the library". The
former contains code derived from the library, whereas the latter must
be combined with the library in order to run.
GNU LESSER GENERAL PUBLIC LICENSE
TERMS AND CONDITIONS FOR COPYING, DISTRIBUTION AND MODIFICATION
0. This License Agreement applies to any software library or other
program which contains a notice placed by the copyright holder or
other authorized party saying it may be distributed under the terms of
this Lesser General Public License (also called "this License").
Each licensee is addressed as "you".
A "library" means a collection of software functions and/or data
prepared so as to be conveniently linked with application programs
(which use some of those functions and data) to form executables.
The "Library", below, refers to any such software library or work
which has been distributed under these terms. A "work based on the
Library" means either the Library or any derivative work under
copyright law: that is to say, a work containing the Library or a
portion of it, either verbatim or with modifications and/or translated
straightforwardly into another language. (Hereinafter, translation is
included without limitation in the term "modification".)
"Source code" for a work means the preferred form of the work for
making modifications to it. For a library, complete source code means
all the source code for all modules it contains, plus any associated
interface definition files, plus the scripts used to control compilation
and installation of the library.
Activities other than copying, distribution and modification are not
covered by this License; they are outside its scope. The act of
running a program using the Library is not restricted, and output from
such a program is covered only if its contents constitute a work based
on the Library (independent of the use of the Library in a tool for
writing it). Whether that is true depends on what the Library does
and what the program that uses the Library does.
1. You may copy and distribute verbatim copies of the Library's
complete source code as you receive it, in any medium, provided that
you conspicuously and appropriately publish on each copy an
appropriate copyright notice and disclaimer of warranty; keep intact
all the notices that refer to this License and to the absence of any
warranty; and distribute a copy of this License along with the
Library.
You may charge a fee for the physical act of transferring a copy,
and you may at your option offer warranty protection in exchange for a
fee.
2. You may modify your copy or copies of the Library or any portion
of it, thus forming a work based on the Library, and copy and
distribute such modifications or work under the terms of Section 1
above, provided that you also meet all of these conditions:
a) The modified work must itself be a software library.
b) You must cause the files modified to carry prominent notices
stating that you changed the files and the date of any change.
c) You must cause the whole of the work to be licensed at no
charge to all third parties under the terms of this License.
d) If a facility in the modified Library refers to a function or a
table of data to be supplied by an application program that uses
the facility, other than as an argument passed when the facility
is invoked, then you must make a good faith effort to ensure that,
in the event an application does not supply such function or
table, the facility still operates, and performs whatever part of
its purpose remains meaningful.
(For example, a function in a library to compute square roots has
a purpose that is entirely well-defined independent of the
application. Therefore, Subsection 2d requires that any
application-supplied function or table used by this function must
be optional: if the application does not supply it, the square
root function must still compute square roots.)
These requirements apply to the modified work as a whole. If
identifiable sections of that work are not derived from the Library,
and can be reasonably considered independent and separate works in
themselves, then this License, and its terms, do not apply to those
sections when you distribute them as separate works. But when you
distribute the same sections as part of a whole which is a work based
on the Library, the distribution of the whole must be on the terms of
this License, whose permissions for other licensees extend to the
entire whole, and thus to each and every part regardless of who wrote
it.
Thus, it is not the intent of this section to claim rights or contest
your rights to work written entirely by you; rather, the intent is to
exercise the right to control the distribution of derivative or
collective works based on the Library.
In addition, mere aggregation of another work not based on the Library
with the Library (or with a work based on the Library) on a volume of
a storage or distribution medium does not bring the other work under
the scope of this License.
3. You may opt to apply the terms of the ordinary GNU General Public
License instead of this License to a given copy of the Library. To do
this, you must alter all the notices that refer to this License, so
that they refer to the ordinary GNU General Public License, version 2,
instead of to this License. (If a newer version than version 2 of the
ordinary GNU General Public License has appeared, then you can specify
that version instead if you wish.) Do not make any other change in
these notices.
Once this change is made in a given copy, it is irreversible for
that copy, so the ordinary GNU General Public License applies to all
subsequent copies and derivative works made from that copy.
This option is useful when you wish to copy part of the code of
the Library into a program that is not a library.
4. You may copy and distribute the Library (or a portion or
derivative of it, under Section 2) in object code or executable form
under the terms of Sections 1 and 2 above provided that you accompany
it with the complete corresponding machine-readable source code, which
must be distributed under the terms of Sections 1 and 2 above on a
medium customarily used for software interchange.
If distribution of object code is made by offering access to copy
from a designated place, then offering equivalent access to copy the
source code from the same place satisfies the requirement to
distribute the source code, even though third parties are not
compelled to copy the source along with the object code.
5. A program that contains no derivative of any portion of the
Library, but is designed to work with the Library by being compiled or
linked with it, is called a "work that uses the Library". Such a
work, in isolation, is not a derivative work of the Library, and
therefore falls outside the scope of this License.
However, linking a "work that uses the Library" with the Library
creates an executable that is a derivative of the Library (because it
contains portions of the Library), rather than a "work that uses the
library". The executable is therefore covered by this License.
Section 6 states terms for distribution of such executables.
When a "work that uses the Library" uses material from a header file
that is part of the Library, the object code for the work may be a
derivative work of the Library even though the source code is not.
Whether this is true is especially significant if the work can be
linked without the Library, or if the work is itself a library. The
threshold for this to be true is not precisely defined by law.
If such an object file uses only numerical parameters, data
structure layouts and accessors, and small macros and small inline
functions (ten lines or less in length), then the use of the object
file is unrestricted, regardless of whether it is legally a derivative
work. (Executables containing this object code plus portions of the
Library will still fall under Section 6.)
Otherwise, if the work is a derivative of the Library, you may
distribute the object code for the work under the terms of Section 6.
Any executables containing that work also fall under Section 6,
whether or not they are linked directly with the Library itself.
6. As an exception to the Sections above, you may also combine or
link a "work that uses the Library" with the Library to produce a
work containing portions of the Library, and distribute that work
under terms of your choice, provided that the terms permit
modification of the work for the customer's own use and reverse
engineering for debugging such modifications.
You must give prominent notice with each copy of the work that the
Library is used in it and that the Library and its use are covered by
this License. You must supply a copy of this License. If the work
during execution displays copyright notices, you must include the
copyright notice for the Library among them, as well as a reference
directing the user to the copy of this License. Also, you must do one
of these things:
a) Accompany the work with the complete corresponding
machine-readable source code for the Library including whatever
changes were used in the work (which must be distributed under
Sections 1 and 2 above); and, if the work is an executable linked
with the Library, with the complete machine-readable "work that
uses the Library", as object code and/or source code, so that the
user can modify the Library and then relink to produce a modified
executable containing the modified Library. (It is understood
that the user who changes the contents of definitions files in the
Library will not necessarily be able to recompile the application
to use the modified definitions.)
b) Use a suitable shared library mechanism for linking with the
Library. A suitable mechanism is one that (1) uses at run time a
copy of the library already present on the user's computer system,
rather than copying library functions into the executable, and (2)
will operate properly with a modified version of the library, if
the user installs one, as long as the modified version is
interface-compatible with the version that the work was made with.
c) Accompany the work with a written offer, valid for at
least three years, to give the same user the materials
specified in Subsection 6a, above, for a charge no more
than the cost of performing this distribution.
d) If distribution of the work is made by offering access to copy
from a designated place, offer equivalent access to copy the above
specified materials from the same place.
e) Verify that the user has already received a copy of these
materials or that you have already sent this user a copy.
For an executable, the required form of the "work that uses the
Library" must include any data and utility programs needed for
reproducing the executable from it. However, as a special exception,
the materials to be distributed need not include anything that is
normally distributed (in either source or binary form) with the major
components (compiler, kernel, and so on) of the operating system on
which the executable runs, unless that component itself accompanies
the executable.
It may happen that this requirement contradicts the license
restrictions of other proprietary libraries that do not normally
accompany the operating system. Such a contradiction means you cannot
use both them and the Library together in an executable that you
distribute.
7. You may place library facilities that are a work based on the
Library side-by-side in a single library together with other library
facilities not covered by this License, and distribute such a combined
library, provided that the separate distribution of the work based on
the Library and of the other library facilities is otherwise
permitted, and provided that you do these two things:
a) Accompany the combined library with a copy of the same work
based on the Library, uncombined with any other library
facilities. This must be distributed under the terms of the
Sections above.
b) Give prominent notice with the combined library of the fact
that part of it is a work based on the Library, and explaining
where to find the accompanying uncombined form of the same work.
8. You may not copy, modify, sublicense, link with, or distribute
the Library except as expressly provided under this License. Any
attempt otherwise to copy, modify, sublicense, link with, or
distribute the Library is void, and will automatically terminate your
rights under this License. However, parties who have received copies,
or rights, from you under this License will not have their licenses
terminated so long as such parties remain in full compliance.
9. You are not required to accept this License, since you have not
signed it. However, nothing else grants you permission to modify or
distribute the Library or its derivative works. These actions are
prohibited by law if you do not accept this License. Therefore, by
modifying or distributing the Library (or any work based on the
Library), you indicate your acceptance of this License to do so, and
all its terms and conditions for copying, distributing or modifying
the Library or works based on it.
10. Each time you redistribute the Library (or any work based on the
Library), the recipient automatically receives a license from the
original licensor to copy, distribute, link with or modify the Library
subject to these terms and conditions. You may not impose any further
restrictions on the recipients' exercise of the rights granted herein.
You are not responsible for enforcing compliance by third parties with
this License.
11. If, as a consequence of a court judgment or allegation of patent
infringement or for any other reason (not limited to patent issues),
conditions are imposed on you (whether by court order, agreement or
otherwise) that contradict the conditions of this License, they do not
excuse you from the conditions of this License. If you cannot
distribute so as to satisfy simultaneously your obligations under this
License and any other pertinent obligations, then as a consequence you
may not distribute the Library at all. For example, if a patent
license would not permit royalty-free redistribution of the Library by
all those who receive copies directly or indirectly through you, then
the only way you could satisfy both it and this License would be to
refrain entirely from distribution of the Library.
If any portion of this section is held invalid or unenforceable under any
particular circumstance, the balance of the section is intended to apply,
and the section as a whole is intended to apply in other circumstances.
It is not the purpose of this section to induce you to infringe any
patents or other property right claims or to contest validity of any
such claims; this section has the sole purpose of protecting the
integrity of the free software distribution system which is
implemented by public license practices. Many people have made
generous contributions to the wide range of software distributed
through that system in reliance on consistent application of that
system; it is up to the author/donor to decide if he or she is willing
to distribute software through any other system and a licensee cannot
impose that choice.
This section is intended to make thoroughly clear what is believed to
be a consequence of the rest of this License.
12. If the distribution and/or use of the Library is restricted in
certain countries either by patents or by copyrighted interfaces, the
original copyright holder who places the Library under this License may add
an explicit geographical distribution limitation excluding those countries,
so that distribution is permitted only in or among countries not thus
excluded. In such case, this License incorporates the limitation as if
written in the body of this License.
13. The Free Software Foundation may publish revised and/or new
versions of the Lesser General Public License from time to time.
Such new versions will be similar in spirit to the present version,
but may differ in detail to address new problems or concerns.
Each version is given a distinguishing version number. If the Library
specifies a version number of this License which applies to it and
"any later version", you have the option of following the terms and
conditions either of that version or of any later version published by
the Free Software Foundation. If the Library does not specify a
license version number, you may choose any version ever published by
the Free Software Foundation.
14. If you wish to incorporate parts of the Library into other free
programs whose distribution conditions are incompatible with these,
write to the author to ask for permission. For software which is
copyrighted by the Free Software Foundation, write to the Free
Software Foundation; we sometimes make exceptions for this. Our
decision will be guided by the two goals of preserving the free status
of all derivatives of our free software and of promoting the sharing
and reuse of software generally.
NO WARRANTY
15. BECAUSE THE LIBRARY IS LICENSED FREE OF CHARGE, THERE IS NO
WARRANTY FOR THE LIBRARY, TO THE EXTENT PERMITTED BY APPLICABLE LAW.
EXCEPT WHEN OTHERWISE STATED IN WRITING THE COPYRIGHT HOLDERS AND/OR
OTHER PARTIES PROVIDE THE LIBRARY "AS IS" WITHOUT WARRANTY OF ANY
KIND, EITHER EXPRESSED OR IMPLIED, INCLUDING, BUT NOT LIMITED TO, THE
IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR
PURPOSE. THE ENTIRE RISK AS TO THE QUALITY AND PERFORMANCE OF THE
LIBRARY IS WITH YOU. SHOULD THE LIBRARY PROVE DEFECTIVE, YOU ASSUME
THE COST OF ALL NECESSARY SERVICING, REPAIR OR CORRECTION.
16. IN NO EVENT UNLESS REQUIRED BY APPLICABLE LAW OR AGREED TO IN
WRITING WILL ANY COPYRIGHT HOLDER, OR ANY OTHER PARTY WHO MAY MODIFY
AND/OR REDISTRIBUTE THE LIBRARY AS PERMITTED ABOVE, BE LIABLE TO YOU
FOR DAMAGES, INCLUDING ANY GENERAL, SPECIAL, INCIDENTAL OR
CONSEQUENTIAL DAMAGES ARISING OUT OF THE USE OR INABILITY TO USE THE
LIBRARY (INCLUDING BUT NOT LIMITED TO LOSS OF DATA OR DATA BEING
RENDERED INACCURATE OR LOSSES SUSTAINED BY YOU OR THIRD PARTIES OR A
FAILURE OF THE LIBRARY TO OPERATE WITH ANY OTHER SOFTWARE), EVEN IF
SUCH HOLDER OR OTHER PARTY HAS BEEN ADVISED OF THE POSSIBILITY OF SUCH
DAMAGES.
END OF TERMS AND CONDITIONS
How to Apply These Terms to Your New Libraries
If you develop a new library, and you want it to be of the greatest
possible use to the public, we recommend making it free software that
everyone can redistribute and change. You can do so by permitting
redistribution under these terms (or, alternatively, under the terms of the
ordinary General Public License).
To apply these terms, attach the following notices to the library. It is
safest to attach them to the start of each source file to most effectively
convey the exclusion of warranty; and each file should have at least the
"copyright" line and a pointer to where the full notice is found.
<one line to give the library's name and a brief idea of what it does.>
Copyright (C) <year> <name of author>
This library is free software; you can redistribute it and/or
modify it under the terms of the GNU Lesser General Public
License as published by the Free Software Foundation; either
version 2.1 of the License, or (at your option) any later version.
This library is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
Lesser General Public License for more details.
You should have received a copy of the GNU Lesser General Public
License along with this library; if not, write to the Free Software
Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA
Also add information on how to contact you by electronic and paper mail.
You should also get your employer (if you work as a programmer) or your
school, if any, to sign a "copyright disclaimer" for the library, if
necessary. Here is a sample; alter the names:
Yoyodyne, Inc., hereby disclaims all copyright interest in the
library `Frob' (a library for tweaking knobs) written by James Random Hacker.
<signature of Ty Coon>, 1 April 1990
Ty Coon, President of Vice
That's all there is to it!
+123
View File
@@ -0,0 +1,123 @@
= CodeRay
Tired of blue'n'gray? Try the original version of this documentation on
coderay.rubychan.de[http://coderay.rubychan.de/doc/] :-)
== About
CodeRay is a Ruby library for syntax highlighting.
You put your code in, and you get it back colored; Keywords, strings,
floats, comments - all in different colors. And with line numbers.
*Syntax* *Highlighting*...
* makes code easier to read and maintain
* lets you detect syntax errors faster
* helps you to understand the syntax of a language
* looks nice
* is what everybody wants to have on their website
* solves all your problems and makes the girls run after you
== Installation
% gem install coderay
=== Dependencies
CodeRay needs Ruby 1.8.7+ or 1.9.2+. It also runs on Rubinius and JRuby.
== Example Usage
require 'coderay'
html = CodeRay.scan("puts 'Hello, world!'", :ruby).div(:line_numbers => :table)
== Documentation
See CodeRay.
== Credits
=== Special Thanks to
* licenser (Heinz N. Gies) for ending my QBasic career, inventing the Coder
project and the input/output plugin system.
CodeRay would not exist without him.
* bovi (Daniel Bovensiepen) for helping me out on various occasions.
=== Thanks to
* Caleb Clausen for writing RubyLexer (see
http://rubyforge.org/projects/rubylexer) and lots of very interesting mail
traffic
* birkenfeld (Georg Brandl) and mitsuhiku (Arnim Ronacher) for PyKleur, now pygments.
You guys rock!
* Jamis Buck for writing Syntax (see http://rubyforge.org/projects/syntax)
I got some useful ideas from it.
* Doug Kearns and everyone else who worked on ruby.vim - it not only helped me
coding CodeRay, but also gave me a wonderful target to reach for the Ruby
scanner.
* everyone who uses CodeBB on http://www.rubyforen.de and http://www.python-forum.de
* iGEL, magichisoka, manveru, WoNáDo and everyone I forgot from rubyforen.de
* Dethix from ruby-mine.de
* zickzackw
* Dookie (who is no longer with us...) and Leonidas from http://www.python-forum.de
* Andreas Schwarz for finding out that CaseIgnoringWordList was not case
ignoring! Such things really make you write tests.
* closure for the first version of the Scheme scanner.
* Stefan Walk for the first version of the JavaScript and PHP scanners.
* Josh Goebel for another version of the JavaScript scanner, a SQL and a Diff scanner.
* Jonathan Younger for pointing out the licence confusion caused by wrong LICENSE file.
* Jeremy Hinegardner for finding the shebang-on-empty-file bug in FileType.
* Charles Oliver Nutter and Yehuda Katz for helping me benchmark CodeRay on JRuby.
* Andreas Neuhaus for pointing out a markup bug in coderay/for_redcloth.
* 0xf30fc7 for the FileType patch concerning Delphi file extensions.
* The folks at redmine.org - thank you for using and fixing CodeRay!
* Keith Pitt for his SQL scanners
* Rob Aldred for the terminal encoder
* Trans for pointing out $DEBUG dependencies
* Flameeyes for finding that Term::ANSIColor was obsolete
* matz and all Ruby gods and gurus
* The inventors of: the computer, the internet, the true color display, HTML &
CSS, VIM, Ruby, pizza, microwaves, guitars, scouting, programming, anime,
manga, coke and green ice tea.
Where would we be without all those people?
=== Created using
* Ruby[http://ruby-lang.org/]
* Chihiro (my Sony VAIO laptop); Henrietta (my old MacBook);
Triella, born Rico (my new MacBook); as well as
Seras and Hikari (my PCs)
* RDE[http://homepage2.nifty.com/sakazuki/rde_e.html],
VIM[http://vim.org] and TextMate[http://macromates.com]
* Subversion[http://subversion.tigris.org/]
* Redmine[http://redmine.org/]
* Firefox[http://www.mozilla.org/products/firefox/],
Firebug[http://getfirebug.com/], Safari[http://www.apple.com/safari/], and
Thunderbird[http://www.mozilla.org/products/thunderbird/]
* RubyGems[http://docs.rubygems.org/] and Rake[http://rake.rubyforge.org/]
* TortoiseSVN[http://tortoisesvn.tigris.org/] using Apache via
XAMPP[http://www.apachefriends.org/en/xampp.html]
* RDoc (though I'm quite unsatisfied with it)
* Microsoft Windows (yes, I confess!) and MacOS X
* GNUWin32, MinGW and some other tools to make the shell under windows a bit
less useless
* Term::ANSIColor[http://term-ansicolor.rubyforge.org/]
* PLEAC[http://pleac.sourceforge.net/] code examples
* Github
* Travis CI (http://travis-ci.org/rubychan/github)
=== Free
* As you can see, CodeRay was created under heavy use of *free* software.
* So CodeRay is also *free*.
* If you use CodeRay to create software, think about making this software
*free*, too.
* Thanks :)
+35
View File
@@ -0,0 +1,35 @@
$:.unshift File.dirname(__FILE__) unless $:.include? '.'
ROOT = '.'
LIB_ROOT = File.join ROOT, 'lib'
task :default => :test
if File.directory? 'rake_tasks'
# load rake tasks from subfolder
for task_file in Dir['rake_tasks/*.rake'].sort
load task_file
end
else
# fallback tasks when rake_tasks folder is not present (eg. in the distribution package)
desc 'Run CodeRay tests (basic)'
task :test do
ruby './test/functional/suite.rb'
ruby './test/functional/for_redcloth.rb'
end
gem 'rdoc' if defined? gem
require 'rdoc/task'
desc 'Generate documentation for CodeRay'
Rake::RDocTask.new :doc do |rd|
rd.title = 'CodeRay Documentation'
rd.main = 'README_INDEX.rdoc'
rd.rdoc_files.add Dir['lib']
rd.rdoc_files.add rd.main
rd.rdoc_dir = 'doc'
end
end
+81
View File
@@ -0,0 +1,81 @@
module CodeRay
# = Duo
#
# A Duo is a convenient way to use CodeRay. You just create a Duo,
# giving it a lang (language of the input code) and a format (desired
# output format), and call Duo#highlight with the code.
#
# Duo makes it easy to re-use both scanner and encoder for a repetitive
# task. It also provides a very easy interface syntax:
#
# require 'coderay'
# CodeRay::Duo[:python, :div].highlight 'import this'
#
# Until you want to do uncommon things with CodeRay, I recommend to use
# this method, since it takes care of everything.
class Duo
attr_accessor :lang, :format, :options
# Create a new Duo, holding a lang and a format to highlight code.
#
# simple:
# CodeRay::Duo[:ruby, :html].highlight 'bla 42'
#
# with options:
# CodeRay::Duo[:ruby, :html, :hint => :debug].highlight '????::??'
#
# alternative syntax without options:
# CodeRay::Duo[:ruby => :statistic].encode 'class << self; end'
#
# alternative syntax with options:
# CodeRay::Duo[{ :ruby => :statistic }, :do => :something].encode 'abc'
#
# The options are forwarded to scanner and encoder
# (see CodeRay.get_scanner_options).
def initialize lang = nil, format = nil, options = {}
if format.nil? && lang.is_a?(Hash) && lang.size == 1
@lang = lang.keys.first
@format = lang[@lang]
else
@lang = lang
@format = format
end
@options = options
end
class << self
# To allow calls like Duo[:ruby, :html].highlight.
alias [] new
end
# The scanner of the duo. Only created once.
def scanner
@scanner ||= CodeRay.scanner @lang, CodeRay.get_scanner_options(@options)
end
# The encoder of the duo. Only created once.
def encoder
@encoder ||= CodeRay.encoder @format, @options
end
# Tokenize and highlight the code using +scanner+ and +encoder+.
def encode code, options = {}
options = @options.merge options
encoder.encode(code, @lang, options)
end
alias highlight encode
# Allows to use Duo like a proc object:
#
# CodeRay::Duo[:python => :yaml].call(code)
#
# or, in Ruby 1.9 and later:
#
# CodeRay::Duo[:python => :yaml].(code)
alias call encode
end
end
+201
View File
@@ -0,0 +1,201 @@
module CodeRay
# This module holds the Encoder class and its subclasses.
# For example, the HTML encoder is named CodeRay::Encoders::HTML
# can be found in coderay/encoders/html.
#
# Encoders also provides methods and constants for the register
# mechanism and the [] method that returns the Encoder class
# belonging to the given format.
module Encoders
extend PluginHost
plugin_path File.dirname(__FILE__), 'encoders'
# = Encoder
#
# The Encoder base class. Together with Scanner and
# Tokens, it forms the highlighting triad.
#
# Encoder instances take a Tokens object and do something with it.
#
# The most common Encoder is surely the HTML encoder
# (CodeRay::Encoders::HTML). It highlights the code in a colorful
# html page.
# If you want the highlighted code in a div or a span instead,
# use its subclasses Div and Span.
class Encoder
extend Plugin
plugin_host Encoders
class << self
# If FILE_EXTENSION isn't defined, this method returns the
# downcase class name instead.
def const_missing sym
if sym == :FILE_EXTENSION
(defined?(@plugin_id) && @plugin_id || name[/\w+$/].downcase).to_s
else
super
end
end
# The default file extension for output file of this encoder class.
def file_extension
self::FILE_EXTENSION
end
end
# Subclasses are to store their default options in this constant.
DEFAULT_OPTIONS = { }
# The options you gave the Encoder at creating.
attr_accessor :options, :scanner
# Creates a new Encoder.
# +options+ is saved and used for all encode operations, as long
# as you don't overwrite it there by passing additional options.
#
# Encoder objects provide three encode methods:
# - encode simply takes a +code+ string and a +lang+
# - encode_tokens expects a +tokens+ object instead
#
# Each method has an optional +options+ parameter. These are
# added to the options you passed at creation.
def initialize options = {}
@options = self.class::DEFAULT_OPTIONS.merge options
@@CODERAY_TOKEN_INTERFACE_DEPRECATION_WARNING_GIVEN = false
end
# Encode a Tokens object.
def encode_tokens tokens, options = {}
options = @options.merge options
@scanner = tokens.scanner if tokens.respond_to? :scanner
setup options
compile tokens, options
finish options
end
# Encode the given +code+ using the Scanner for +lang+.
def encode code, lang, options = {}
options = @options.merge options
@scanner = Scanners[lang].new code, CodeRay.get_scanner_options(options).update(:tokens => self)
setup options
@scanner.tokenize
finish options
end
# You can use highlight instead of encode, if that seems
# more clear to you.
alias highlight encode
# The default file extension for this encoder.
def file_extension
self.class.file_extension
end
def << token
unless @@CODERAY_TOKEN_INTERFACE_DEPRECATION_WARNING_GIVEN
warn 'Using old Tokens#<< interface.'
@@CODERAY_TOKEN_INTERFACE_DEPRECATION_WARNING_GIVEN = true
end
self.token(*token)
end
# Called with +content+ and +kind+ of the currently scanned token.
# For simple scanners, it's enougth to implement this method.
#
# By default, it calls text_token, begin_group, end_group, begin_line,
# or end_line, depending on the +content+.
def token content, kind
case content
when String
text_token content, kind
when :begin_group
begin_group kind
when :end_group
end_group kind
when :begin_line
begin_line kind
when :end_line
end_line kind
else
raise ArgumentError, 'Unknown token content type: %p, kind = %p' % [content, kind]
end
end
# Called for each text token ([text, kind]), where text is a String.
def text_token text, kind
@out << text
end
# Starts a token group with the given +kind+.
def begin_group kind
end
# Ends a token group with the given +kind+.
def end_group kind
end
# Starts a new line token group with the given +kind+.
def begin_line kind
end
# Ends a new line token group with the given +kind+.
def end_line kind
end
protected
# Called with merged options before encoding starts.
# Sets @out to an empty string.
#
# See the HTML Encoder for an example of option caching.
def setup options
@out = get_output(options)
end
def get_output options
options[:out] || ''
end
# Append data.to_s to the output. Returns the argument.
def output data
@out << data.to_s
data
end
# Called with merged options after encoding starts.
# The return value is the result of encoding, typically @out.
def finish options
@out
end
# Do the encoding.
#
# The already created +tokens+ object must be used; it must be a
# Tokens object.
def compile tokens, options = {}
content = nil
for item in tokens
if item.is_a? Array
raise ArgumentError, 'Two-element array tokens are no longer supported.'
end
if content
token content, item
content = nil
else
content = item
end
end
raise 'odd number list for Tokens' if content
end
alias tokens compile
public :tokens
end
end
end
+17
View File
@@ -0,0 +1,17 @@
module CodeRay
module Encoders
map \
:loc => :lines_of_code,
:plain => :text,
:plaintext => :text,
:remove_comments => :comment_filter,
:stats => :statistic,
:term => :terminal,
:tty => :terminal,
:yml => :yaml
# No default because Tokens#nonsense should raise NoMethodError.
end
end
@@ -0,0 +1,25 @@
module CodeRay
module Encoders
load :token_kind_filter
# A simple Filter that removes all tokens of the :comment kind.
#
# Alias: +remove_comments+
#
# Usage:
# CodeRay.scan('print # foo', :ruby).comment_filter.text
# #-> "print "
#
# See also: TokenKindFilter, LinesOfCode
class CommentFilter < TokenKindFilter
register_for :comment_filter
DEFAULT_OPTIONS = superclass::DEFAULT_OPTIONS.merge \
:exclude => [:comment, :docstring]
end
end
end
+39
View File
@@ -0,0 +1,39 @@
module CodeRay
module Encoders
# Returns the number of tokens.
#
# Text and block tokens are counted.
class Count < Encoder
register_for :count
protected
def setup options
super
@count = 0
end
def finish options
output @count
end
public
def text_token text, kind
@count += 1
end
def begin_group kind
@count += 1
end
alias end_group begin_group
alias begin_line begin_group
alias end_line begin_group
end
end
end
+61
View File
@@ -0,0 +1,61 @@
module CodeRay
module Encoders
# = Debug Encoder
#
# Fast encoder producing simple debug output.
#
# It is readable and diff-able and is used for testing.
#
# You cannot fully restore the tokens information from the
# output, because consecutive :space tokens are merged.
# Use Tokens#dump for caching purposes.
#
# See also: Scanners::Debug
class Debug < Encoder
register_for :debug
FILE_EXTENSION = 'raydebug'
def initialize options = {}
super
@opened = []
end
def text_token text, kind
if kind == :space
@out << text
else
# TODO: Escape (
text = text.gsub(/[)\\]/, '\\\\\0') # escape ) and \
@out << kind.to_s << '(' << text << ')'
end
end
def begin_group kind
@opened << kind
@out << kind.to_s << '<'
end
def end_group kind
if @opened.last != kind
puts @out
raise "we are inside #{@opened.inspect}, not #{kind}"
end
@opened.pop
@out << '>'
end
def begin_line kind
@out << kind.to_s << '['
end
def end_line kind
@out << ']'
end
end
end
end
+23
View File
@@ -0,0 +1,23 @@
module CodeRay
module Encoders
load :html
# Wraps HTML output into a DIV element, using inline styles by default.
#
# See Encoders::HTML for available options.
class Div < HTML
FILE_EXTENSION = 'div.html'
register_for :div
DEFAULT_OPTIONS = HTML::DEFAULT_OPTIONS.merge \
:css => :style,
:wrap => :div,
:line_numbers => false
end
end
end
@@ -0,0 +1,58 @@
module CodeRay
module Encoders
# A Filter encoder has another Tokens instance as output.
# It can be subclass to select, remove, or modify tokens in the stream.
#
# Subclasses of Filter are called "Filters" and can be chained.
#
# == Options
#
# === :tokens
#
# The Tokens object which will receive the output.
#
# Default: Tokens.new
#
# See also: TokenKindFilter
class Filter < Encoder
register_for :filter
protected
def setup options
super
@tokens = options[:tokens] || Tokens.new
end
def finish options
output @tokens
end
public
def text_token text, kind # :nodoc:
@tokens.text_token text, kind
end
def begin_group kind # :nodoc:
@tokens.begin_group kind
end
def begin_line kind # :nodoc:
@tokens.begin_line kind
end
def end_group kind # :nodoc:
@tokens.end_group kind
end
def end_line kind # :nodoc:
@tokens.end_line kind
end
end
end
end
+302
View File
@@ -0,0 +1,302 @@
require 'set'
module CodeRay
module Encoders
# = HTML Encoder
#
# This is CodeRay's most important highlighter:
# It provides save, fast XHTML generation and CSS support.
#
# == Usage
#
# require 'coderay'
# puts CodeRay.scan('Some /code/', :ruby).html #-> a HTML page
# puts CodeRay.scan('Some /code/', :ruby).html(:wrap => :span)
# #-> <span class="CodeRay"><span class="co">Some</span> /code/</span>
# puts CodeRay.scan('Some /code/', :ruby).span #-> the same
#
# puts CodeRay.scan('Some code', :ruby).html(
# :wrap => nil,
# :line_numbers => :inline,
# :css => :style
# )
#
# == Options
#
# === :tab_width
# Convert \t characters to +n+ spaces (a number.)
#
# Default: 8
#
# === :css
# How to include the styles; can be :class or :style.
#
# Default: :class
#
# === :wrap
# Wrap in :page, :div, :span or nil.
#
# You can also use Encoders::Div and Encoders::Span.
#
# Default: nil
#
# === :title
#
# The title of the HTML page (works only when :wrap is set to :page.)
#
# Default: 'CodeRay output'
#
# === :line_numbers
# Include line numbers in :table, :inline, or nil (no line numbers)
#
# Default: nil
#
# === :line_number_anchors
# Adds anchors and links to the line numbers. Can be false (off), true (on),
# or a prefix string that will be prepended to the anchor name.
#
# The prefix must consist only of letters, digits, and underscores.
#
# Default: true, default prefix name: "line"
#
# === :line_number_start
# Where to start with line number counting.
#
# Default: 1
#
# === :bold_every
# Make every +n+-th number appear bold.
#
# Default: 10
#
# === :highlight_lines
#
# Highlights certain line numbers.
# Can be any Enumerable, typically just an Array or Range, of numbers.
#
# Bolding is deactivated when :highlight_lines is set. It only makes sense
# in combination with :line_numbers.
#
# Default: nil
#
# === :hint
# Include some information into the output using the title attribute.
# Can be :info (show token kind on mouse-over), :info_long (with full path)
# or :debug (via inspect).
#
# Default: false
class HTML < Encoder
register_for :html
FILE_EXTENSION = 'snippet.html'
DEFAULT_OPTIONS = {
:tab_width => 8,
:css => :class,
:style => :alpha,
:wrap => nil,
:title => 'CodeRay output',
:line_numbers => nil,
:line_number_anchors => 'n',
:line_number_start => 1,
:bold_every => 10,
:highlight_lines => nil,
:hint => false,
}
autoload :Output, 'coderay/encoders/html/output'
autoload :CSS, 'coderay/encoders/html/css'
autoload :Numbering, 'coderay/encoders/html/numbering'
attr_reader :css
protected
HTML_ESCAPE = { #:nodoc:
'&' => '&amp;',
'"' => '&quot;',
'>' => '&gt;',
'<' => '&lt;',
}
# This was to prevent illegal HTML.
# Strange chars should still be avoided in codes.
evil_chars = Array(0x00...0x20) - [?\n, ?\t, ?\s]
evil_chars.each { |i| HTML_ESCAPE[i.chr] = ' ' }
#ansi_chars = Array(0x7f..0xff)
#ansi_chars.each { |i| HTML_ESCAPE[i.chr] = '&#%d;' % i }
# \x9 (\t) and \xA (\n) not included
#HTML_ESCAPE_PATTERN = /[\t&"><\0-\x8\xB-\x1f\x7f-\xff]/
HTML_ESCAPE_PATTERN = /[\t"&><\0-\x8\xB-\x1f]/
TOKEN_KIND_TO_INFO = Hash.new do |h, kind|
h[kind] = kind.to_s.gsub(/_/, ' ').gsub(/\b\w/) { $&.capitalize }
end
TRANSPARENT_TOKEN_KINDS = Set[
:delimiter, :modifier, :content, :escape, :inline_delimiter,
]
# Generate a hint about the given +kinds+ in a +hint+ style.
#
# +hint+ may be :info, :info_long or :debug.
def self.token_path_to_hint hint, kinds
kinds = Array kinds
title =
case hint
when :info
kinds = kinds[1..-1] if TRANSPARENT_TOKEN_KINDS.include? kinds.first
TOKEN_KIND_TO_INFO[kinds.first]
when :info_long
kinds.reverse.map { |kind| TOKEN_KIND_TO_INFO[kind] }.join('/')
when :debug
kinds.inspect
end
title ? " title=\"#{title}\"" : ''
end
def setup options
super
if options[:wrap] || options[:line_numbers]
@real_out = @out
@out = ''
end
@HTML_ESCAPE = HTML_ESCAPE.dup
@HTML_ESCAPE["\t"] = ' ' * options[:tab_width]
@opened = []
@last_opened = nil
@css = CSS.new options[:style]
hint = options[:hint]
if hint && ![:debug, :info, :info_long].include?(hint)
raise ArgumentError, "Unknown value %p for :hint; \
expected :info, :info_long, :debug, false, or nil." % hint
end
css_classes = TokenKinds
case options[:css]
when :class
@span_for_kind = Hash.new do |h, k|
if k.is_a? ::Symbol
kind = k_dup = k
else
kind = k.first
k_dup = k.dup
end
if kind != :space && (hint || css_class = css_classes[kind])
title = HTML.token_path_to_hint hint, k if hint
css_class ||= css_classes[kind]
h[k_dup] = "<span#{title}#{" class=\"#{css_class}\"" if css_class}>"
else
h[k_dup] = nil
end
end
when :style
@span_for_kind = Hash.new do |h, k|
kind = k.is_a?(Symbol) ? k : k.first
h[k.is_a?(Symbol) ? k : k.dup] =
if kind != :space && (hint || css_classes[kind])
title = HTML.token_path_to_hint hint, k if hint
style = @css.get_style Array(k).map { |c| css_classes[c] }
"<span#{title}#{" style=\"#{style}\"" if style}>"
end
end
else
raise ArgumentError, "Unknown value %p for :css." % options[:css]
end
@set_last_opened = options[:hint] || options[:css] == :style
end
def finish options
unless @opened.empty?
warn '%d tokens still open: %p' % [@opened.size, @opened] if $CODERAY_DEBUG
@out << '</span>' while @opened.pop
@last_opened = nil
end
@out.extend Output
@out.css = @css
if options[:line_numbers]
Numbering.number! @out, options[:line_numbers], options
end
@out.wrap! options[:wrap]
@out.apply_title! options[:title]
if defined?(@real_out) && @real_out
@real_out << @out
@out = @real_out
end
super
end
public
def text_token text, kind
if text =~ /#{HTML_ESCAPE_PATTERN}/o
text = text.gsub(/#{HTML_ESCAPE_PATTERN}/o) { |m| @HTML_ESCAPE[m] }
end
if style = @span_for_kind[@last_opened ? [kind, *@opened] : kind]
@out << style << text << '</span>'
else
@out << text
end
end
# token groups, eg. strings
def begin_group kind
@out << (@span_for_kind[@last_opened ? [kind, *@opened] : kind] || '<span>')
@opened << kind
@last_opened = kind if @set_last_opened
end
def end_group kind
if $CODERAY_DEBUG && (@opened.empty? || @opened.last != kind)
warn 'Malformed token stream: Trying to close a token (%p) ' \
'that is not open. Open are: %p.' % [kind, @opened[1..-1]]
end
if @opened.pop
@out << '</span>'
@last_opened = @opened.last if @last_opened
end
end
# whole lines to be highlighted, eg. a deleted line in a diff
def begin_line kind
if style = @span_for_kind[@last_opened ? [kind, *@opened] : kind]
if style['class="']
@out << style.sub('class="', 'class="line ')
else
@out << style.sub('>', ' class="line">')
end
else
@out << '<span class="line">'
end
@opened << kind
@last_opened = kind if @options[:css] == :style
end
def end_line kind
if $CODERAY_DEBUG && (@opened.empty? || @opened.last != kind)
warn 'Malformed token stream: Trying to close a line (%p) ' \
'that is not open. Open are: %p.' % [kind, @opened[1..-1]]
end
if @opened.pop
@out << '</span>'
@last_opened = @opened.last if @last_opened
end
end
end
end
end
+83
View File
@@ -0,0 +1,83 @@
module CodeRay
module Encoders
# A simple JSON Encoder.
#
# Example:
# CodeRay.scan('puts "Hello world!"', :ruby).json
# yields
# [
# {"type"=>"text", "text"=>"puts", "kind"=>"ident"},
# {"type"=>"text", "text"=>" ", "kind"=>"space"},
# {"type"=>"block", "action"=>"open", "kind"=>"string"},
# {"type"=>"text", "text"=>"\"", "kind"=>"delimiter"},
# {"type"=>"text", "text"=>"Hello world!", "kind"=>"content"},
# {"type"=>"text", "text"=>"\"", "kind"=>"delimiter"},
# {"type"=>"block", "action"=>"close", "kind"=>"string"},
# ]
class JSON < Encoder
begin
require 'json'
rescue LoadError
begin
require 'rubygems' unless defined? Gem
gem 'json'
require 'json'
rescue LoadError
$stderr.puts "The JSON encoder needs the JSON library.\n" \
"Please gem install json."
raise
end
end
register_for :json
FILE_EXTENSION = 'json'
protected
def setup options
super
@first = true
@out << '['
end
def finish options
@out << ']'
end
def append data
if @first
@first = false
else
@out << ','
end
@out << data.to_json
end
public
def text_token text, kind
append :type => 'text', :text => text, :kind => kind
end
def begin_group kind
append :type => 'block', :action => 'open', :kind => kind
end
def end_group kind
append :type => 'block', :action => 'close', :kind => kind
end
def begin_line kind
append :type => 'block', :action => 'begin_line', :kind => kind
end
def end_line kind
append :type => 'block', :action => 'end_line', :kind => kind
end
end
end
end
@@ -0,0 +1,45 @@
module CodeRay
module Encoders
# Counts the LoC (Lines of Code). Returns an Integer >= 0.
#
# Alias: +loc+
#
# Everything that is not comment, markup, doctype/shebang, or an empty line,
# is considered to be code.
#
# For example,
# * HTML files not containing JavaScript have 0 LoC
# * in a Java class without comments, LoC is the number of non-empty lines
#
# A Scanner class should define the token kinds that are not code in the
# KINDS_NOT_LOC constant, which defaults to [:comment, :doctype].
class LinesOfCode < TokenKindFilter
register_for :lines_of_code
NON_EMPTY_LINE = /^\s*\S.*$/
protected
def setup options
if scanner
kinds_not_loc = scanner.class::KINDS_NOT_LOC
else
warn "Tokens have no associated scanner, counting all nonempty lines." if $VERBOSE
kinds_not_loc = CodeRay::Scanners::Scanner::KINDS_NOT_LOC
end
options[:exclude] = kinds_not_loc
super options
end
def finish options
output @tokens.text.scan(NON_EMPTY_LINE).size
end
end
end
end
+18
View File
@@ -0,0 +1,18 @@
module CodeRay
module Encoders
# = Null Encoder
#
# Does nothing and returns an empty string.
class Null < Encoder
register_for :null
def text_token text, kind
# do nothing
end
end
end
end
+24
View File
@@ -0,0 +1,24 @@
module CodeRay
module Encoders
load :html
# Wraps the output into a HTML page, using CSS classes and
# line numbers in the table format by default.
#
# See Encoders::HTML for available options.
class Page < HTML
FILE_EXTENSION = 'html'
register_for :page
DEFAULT_OPTIONS = HTML::DEFAULT_OPTIONS.merge \
:css => :class,
:wrap => :page,
:line_numbers => :table
end
end
end
+23
View File
@@ -0,0 +1,23 @@
module CodeRay
module Encoders
load :html
# Wraps HTML output into a SPAN element, using inline styles by default.
#
# See Encoders::HTML for available options.
class Span < HTML
FILE_EXTENSION = 'span.html'
register_for :span
DEFAULT_OPTIONS = HTML::DEFAULT_OPTIONS.merge \
:css => :style,
:wrap => :span,
:line_numbers => false
end
end
end
@@ -0,0 +1,96 @@
module CodeRay
module Encoders
# Makes a statistic for the given tokens.
#
# Alias: +stats+
class Statistic < Encoder
register_for :statistic
attr_reader :type_stats, :real_token_count # :nodoc:
TypeStats = Struct.new :count, :size # :nodoc:
protected
def setup options
super
@type_stats = Hash.new { |h, k| h[k] = TypeStats.new 0, 0 }
@real_token_count = 0
end
STATS = <<-STATS # :nodoc:
Code Statistics
Tokens %8d
Non-Whitespace %8d
Bytes Total %8d
Token Types (%d):
type count ratio size (average)
-------------------------------------------------------------
%s
STATS
TOKEN_TYPES_ROW = <<-TKR # :nodoc:
%-20s %8d %6.2f %% %5.1f
TKR
def finish options
all = @type_stats['TOTAL']
all_count, all_size = all.count, all.size
@type_stats.each do |type, stat|
stat.size /= stat.count.to_f
end
types_stats = @type_stats.sort_by { |k, v| [-v.count, k.to_s] }.map do |k, v|
TOKEN_TYPES_ROW % [k, v.count, 100.0 * v.count / all_count, v.size]
end.join
@out << STATS % [
all_count, @real_token_count, all_size,
@type_stats.delete_if { |k, v| k.is_a? String }.size,
types_stats
]
super
end
public
def text_token text, kind
@real_token_count += 1 unless kind == :space
@type_stats[kind].count += 1
@type_stats[kind].size += text.size
@type_stats['TOTAL'].size += text.size
@type_stats['TOTAL'].count += 1
end
# TODO Hierarchy handling
def begin_group kind
block_token ':begin_group', kind
end
def end_group kind
block_token ':end_group', kind
end
def begin_line kind
block_token ':begin_line', kind
end
def end_line kind
block_token ':end_line', kind
end
def block_token action, kind
@type_stats['TOTAL'].count += 1
@type_stats[action].count += 1
@type_stats[kind].count += 1
end
end
end
end
@@ -0,0 +1,179 @@
module CodeRay
module Encoders
# Outputs code highlighted for a color terminal.
#
# Note: This encoder is in beta. It currently doesn't use the Styles.
#
# Alias: +term+
#
# == Authors & License
#
# By Rob Aldred (http://robaldred.co.uk)
#
# Based on idea by Nathan Weizenbaum (http://nex-3.com)
#
# MIT License (http://www.opensource.org/licenses/mit-license.php)
class Terminal < Encoder
register_for :terminal
TOKEN_COLORS = {
:annotation => '35',
:attribute_name => '33',
:attribute_value => '31',
:binary => '1;35',
:char => {
:self => '36', :delimiter => '34'
},
:class => '1;35',
:class_variable => '36',
:color => '32',
:comment => '37',
:complex => '34',
:constant => ['34', '4'],
:decoration => '35',
:definition => '1;32',
:directive => ['32', '4'],
:doc => '46',
:doctype => '1;30',
:doc_string => ['31', '4'],
:entity => '33',
:error => ['1;33', '41'],
:exception => '1;31',
:float => '1;35',
:function => '1;34',
:global_variable => '42',
:hex => '1;36',
:include => '33',
:integer => '1;34',
:key => '35',
:label => '1;15',
:local_variable => '33',
:octal => '1;35',
:operator_name => '1;29',
:predefined_constant => '1;36',
:predefined_type => '1;30',
:predefined => ['4', '1;34'],
:preprocessor => '36',
:pseudo_class => '34',
:regexp => {
:self => '31',
:content => '31',
:delimiter => '1;29',
:modifier => '35',
:function => '1;29'
},
:reserved => '1;31',
:shell => {
:self => '42',
:content => '1;29',
:delimiter => '37',
},
:string => {
:self => '32',
:modifier => '1;32',
:escape => '1;36',
:delimiter => '1;32',
},
:symbol => '1;32',
:tag => '34',
:type => '1;34',
:value => '36',
:variable => '34',
:insert => '42',
:delete => '41',
:change => '44',
:head => '45'
}
TOKEN_COLORS[:keyword] = TOKEN_COLORS[:reserved]
TOKEN_COLORS[:method] = TOKEN_COLORS[:function]
TOKEN_COLORS[:imaginary] = TOKEN_COLORS[:complex]
TOKEN_COLORS[:begin_group] = TOKEN_COLORS[:end_group] =
TOKEN_COLORS[:escape] = TOKEN_COLORS[:delimiter]
protected
def setup(options)
super
@opened = []
@subcolors = nil
end
public
def text_token text, kind
if color = (@subcolors || TOKEN_COLORS)[kind]
if Hash === color
if color[:self]
color = color[:self]
else
@out << text
return
end
end
@out << ansi_colorize(color)
@out << text.gsub("\n", ansi_clear + "\n" + ansi_colorize(color))
@out << ansi_clear
@out << ansi_colorize(@subcolors[:self]) if @subcolors && @subcolors[:self]
else
@out << text
end
end
def begin_group kind
@opened << kind
@out << open_token(kind)
end
alias begin_line begin_group
def end_group kind
if @opened.empty?
# nothing to close
else
@opened.pop
@out << ansi_clear
@out << open_token(@opened.last)
end
end
def end_line kind
if @opened.empty?
# nothing to close
else
@opened.pop
# whole lines to be highlighted,
# eg. added/modified/deleted lines in a diff
@out << "\t" * 100 + ansi_clear
@out << open_token(@opened.last)
end
end
private
def open_token kind
if color = TOKEN_COLORS[kind]
if Hash === color
@subcolors = color
ansi_colorize(color[:self]) if color[:self]
else
@subcolors = {}
ansi_colorize(color)
end
else
@subcolors = nil
''
end
end
def ansi_colorize(color)
Array(color).map { |c| "\e[#{c}m" }.join
end
def ansi_clear
ansi_colorize(0)
end
end
end
end
+46
View File
@@ -0,0 +1,46 @@
module CodeRay
module Encoders
# Concats the tokens into a single string, resulting in the original
# code string if no tokens were removed.
#
# Alias: +plain+, +plaintext+
#
# == Options
#
# === :separator
# A separator string to join the tokens.
#
# Default: empty String
class Text < Encoder
register_for :text
FILE_EXTENSION = 'txt'
DEFAULT_OPTIONS = {
:separator => nil
}
def text_token text, kind
super
if @first
@first = false
else
@out << @sep
end if @sep
end
protected
def setup options
super
@first = true
@sep = options[:separator]
end
end
end
end
@@ -0,0 +1,111 @@
module CodeRay
module Encoders
load :filter
# A Filter that selects tokens based on their token kind.
#
# == Options
#
# === :exclude
#
# One or many symbols (in an Array) which shall be excluded.
#
# Default: []
#
# === :include
#
# One or many symbols (in an array) which shall be included.
#
# Default: :all, which means all tokens are included.
#
# Exclusion wins over inclusion.
#
# See also: CommentFilter
class TokenKindFilter < Filter
register_for :token_kind_filter
DEFAULT_OPTIONS = {
:exclude => [],
:include => :all
}
protected
def setup options
super
@group_excluded = false
@exclude = options[:exclude]
@exclude = Array(@exclude) unless @exclude == :all
@include = options[:include]
@include = Array(@include) unless @include == :all
end
def include_text_token? text, kind
include_group? kind
end
def include_group? kind
(@include == :all || @include.include?(kind)) &&
!(@exclude == :all || @exclude.include?(kind))
end
public
# Add the token to the output stream if +kind+ matches the conditions.
def text_token text, kind
super if !@group_excluded && include_text_token?(text, kind)
end
# Add the token group to the output stream if +kind+ matches the
# conditions.
#
# If it does not, all tokens inside the group are excluded from the
# stream, even if their kinds match.
def begin_group kind
if @group_excluded
@group_excluded += 1
elsif include_group? kind
super
else
@group_excluded = 1
end
end
# See +begin_group+.
def begin_line kind
if @group_excluded
@group_excluded += 1
elsif include_group? kind
super
else
@group_excluded = 1
end
end
# Take care of re-enabling the delegation of tokens to the output stream
# if an exluded group has ended.
def end_group kind
if @group_excluded
@group_excluded -= 1
@group_excluded = false if @group_excluded.zero?
else
super
end
end
# See +end_group+.
def end_line kind
if @group_excluded
@group_excluded -= 1
@group_excluded = false if @group_excluded.zero?
else
super
end
end
end
end
end
+72
View File
@@ -0,0 +1,72 @@
module CodeRay
module Encoders
# = XML Encoder
#
# Uses REXML. Very slow.
class XML < Encoder
register_for :xml
FILE_EXTENSION = 'xml'
autoload :REXML, 'rexml/document'
DEFAULT_OPTIONS = {
:tab_width => 8,
:pretty => -1,
:transitive => false,
}
protected
def setup options
super
@doc = REXML::Document.new
@doc << REXML::XMLDecl.new
@tab_width = options[:tab_width]
@root = @node = @doc.add_element('coderay-tokens')
end
def finish options
@doc.write @out, options[:pretty], options[:transitive], true
super
end
public
def text_token text, kind
if kind == :space
token = @node
else
token = @node.add_element kind.to_s
end
text.scan(/(\x20+)|(\t+)|(\n)|[^\x20\t\n]+/) do |space, tab, nl|
case
when space
token << REXML::Text.new(space, true)
when tab
token << REXML::Text.new(tab, true)
when nl
token << REXML::Text.new(nl, true)
else
token << REXML::Text.new($&)
end
end
end
def begin_group kind
@node = @node.add_element kind.to_s
end
def end_group kind
if @node == @root
raise 'no token to close!'
end
@node = @node.parent
end
end
end
end
+50
View File
@@ -0,0 +1,50 @@
autoload :YAML, 'yaml'
module CodeRay
module Encoders
# = YAML Encoder
#
# Slow.
class YAML < Encoder
register_for :yaml
FILE_EXTENSION = 'yaml'
protected
def setup options
super
@data = []
end
def finish options
output ::YAML.dump(@data)
end
public
def text_token text, kind
@data << [text, kind]
end
def begin_group kind
@data << [:begin_group, kind]
end
def end_group kind
@data << [:end_group, kind]
end
def begin_line kind
@data << [:begin_line, kind]
end
def end_line kind
@data << [:end_line, kind]
end
end
end
end
+95
View File
@@ -0,0 +1,95 @@
module CodeRay
# A little hack to enable CodeRay highlighting in RedCloth.
#
# Usage:
# require 'coderay'
# require 'coderay/for_redcloth'
# RedCloth.new('@[ruby]puts "Hello, World!"@').to_html
#
# Make sure you have RedCloth 4.0.3 activated, for example by calling
# require 'rubygems'
# before RedCloth is loaded and before calling CodeRay.for_redcloth.
module ForRedCloth
def self.install
gem 'RedCloth', '>= 4.0.3' if defined? gem
require 'redcloth'
unless RedCloth::VERSION.to_s >= '4.0.3'
if defined? gem
raise 'CodeRay.for_redcloth needs RedCloth version 4.0.3 or later. ' +
"You have #{RedCloth::VERSION}. Please gem install RedCloth."
else
$".delete 'redcloth.rb' # sorry, but it works
require 'rubygems'
return install # retry
end
end
unless RedCloth::VERSION.to_s >= '4.2.2'
warn 'CodeRay.for_redcloth works best with RedCloth version 4.2.2 or later.'
end
RedCloth::TextileDoc.send :include, ForRedCloth::TextileDoc
RedCloth::Formatters::HTML.module_eval do
def unescape(html) # :nodoc:
replacements = {
'&amp;' => '&',
'&quot;' => '"',
'&gt;' => '>',
'&lt;' => '<',
}
html.gsub(/&(?:amp|quot|[gl]t);/) { |entity| replacements[entity] }
end
undef code, bc_open, bc_close, escape_pre
def code(opts) # :nodoc:
opts[:block] = true
if !opts[:lang] && RedCloth::VERSION.to_s >= '4.2.0'
# simulating pre-4.2 behavior
if opts[:text].sub!(/\A\[(\w+)\]/, '')
if CodeRay::Scanners[$1].lang == :text
opts[:text] = $& + opts[:text]
else
opts[:lang] = $1
end
end
end
if opts[:lang] && !filter_coderay
require 'coderay'
@in_bc ||= nil
format = @in_bc ? :div : :span
opts[:text] = unescape(opts[:text]) unless @in_bc
highlighted_code = CodeRay.encode opts[:text], opts[:lang], format
highlighted_code.sub!(/\A<(span|div)/) { |m| m + pba(@in_bc || opts) }
highlighted_code
else
"<code#{pba(opts)}>#{opts[:text]}</code>"
end
end
def bc_open(opts) # :nodoc:
opts[:block] = true
@in_bc = opts
opts[:lang] ? '' : "<pre#{pba(opts)}>"
end
def bc_close(opts) # :nodoc:
opts = @in_bc
@in_bc = nil
opts[:lang] ? '' : "</pre>\n"
end
def escape_pre(text) # :nodoc:
if @in_bc ||= nil
text
else
html_esc(text, :html_escape_preformatted)
end
end
end
end
module TextileDoc # :nodoc:
attr_accessor :filter_coderay
end
end
end
CodeRay::ForRedCloth.install
@@ -0,0 +1,141 @@
module CodeRay
# = FileType
#
# A simple filetype recognizer.
#
# == Usage
#
# # determine the type of the given
# lang = FileType[file_name]
#
# # return :text if the file type is unknown
# lang = FileType.fetch file_name, :text
#
# # try the shebang line, too
# lang = FileType.fetch file_name, :text, true
module FileType
UnknownFileType = Class.new Exception
class << self
# Try to determine the file type of the file.
#
# +filename+ is a relative or absolute path to a file.
#
# The file itself is only accessed when +read_shebang+ is set to true.
# That means you can get filetypes from files that don't exist.
def [] filename, read_shebang = false
name = File.basename filename
ext = File.extname(name).sub(/^\./, '') # from last dot, delete the leading dot
ext2 = filename.to_s[/\.(.*)/, 1] # from first dot
type =
TypeFromExt[ext] ||
TypeFromExt[ext.downcase] ||
(TypeFromExt[ext2] if ext2) ||
(TypeFromExt[ext2.downcase] if ext2) ||
TypeFromName[name] ||
TypeFromName[name.downcase]
type ||= shebang(filename) if read_shebang
type
end
# This works like Hash#fetch.
#
# If the filetype cannot be found, the +default+ value
# is returned.
def fetch filename, default = nil, read_shebang = false
if default && block_given?
warn 'Block supersedes default value argument; use either.'
end
if type = self[filename, read_shebang]
type
else
return yield if block_given?
return default if default
raise UnknownFileType, 'Could not determine type of %p.' % filename
end
end
protected
def shebang filename
return unless File.exist? filename
File.open filename, 'r' do |f|
if first_line = f.gets
if type = first_line[TypeFromShebang]
type.to_sym
end
end
end
end
end
TypeFromExt = {
'c' => :c,
'cfc' => :xml,
'cfm' => :xml,
'clj' => :clojure,
'css' => :css,
'diff' => :diff,
'dpr' => :delphi,
'gemspec' => :ruby,
'groovy' => :groovy,
'gvy' => :groovy,
'h' => :c,
'haml' => :haml,
'htm' => :page,
'html' => :page,
'html.erb' => :erb,
'java' => :java,
'js' => :java_script,
'json' => :json,
'mab' => :ruby,
'pas' => :delphi,
'patch' => :diff,
'php' => :php,
'php3' => :php,
'php4' => :php,
'php5' => :php,
'prawn' => :ruby,
'py' => :python,
'py3' => :python,
'pyw' => :python,
'rake' => :ruby,
'raydebug' => :raydebug,
'rb' => :ruby,
'rbw' => :ruby,
'rhtml' => :erb,
'rjs' => :ruby,
'rpdf' => :ruby,
'ru' => :ruby,
'rxml' => :ruby,
# 'sch' => :scheme,
'sql' => :sql,
# 'ss' => :scheme,
'xhtml' => :page,
'xml' => :xml,
'yaml' => :yaml,
'yml' => :yaml,
}
for cpp_alias in %w[cc cpp cp cxx c++ C hh hpp h++ cu]
TypeFromExt[cpp_alias] = :cpp
end
TypeFromShebang = /\b(?:ruby|perl|python|sh)\b/
TypeFromName = {
'Capfile' => :ruby,
'Rakefile' => :ruby,
'Rantfile' => :ruby,
'Gemfile' => :ruby,
}
end
end
+41
View File
@@ -0,0 +1,41 @@
module CodeRay
# A simplified interface to the gzip library +zlib+ (from the Ruby Standard Library.)
module GZip
require 'zlib'
# The default zipping level. 7 zips good and fast.
DEFAULT_GZIP_LEVEL = 7
# Unzips the given string +s+.
#
# Example:
# require 'gzip_simple'
# print GZip.gunzip(File.read('adresses.gz'))
def GZip.gunzip s
Zlib::Inflate.inflate s
end
# Zips the given string +s+.
#
# Example:
# require 'gzip_simple'
# File.open('adresses.gz', 'w') do |file
# file.write GZip.gzip('Mum: 0123 456 789', 9)
# end
#
# If you provide a +level+, you can control how strong
# the string is compressed:
# - 0: no compression, only convert to gzip format
# - 1: compress fast
# - 7: compress more, but still fast (default)
# - 8: compress more, slower
# - 9: compress best, very slow
def GZip.gzip s, level = DEFAULT_GZIP_LEVEL
Zlib::Deflate.new(level).deflate s, Zlib::FINISH
end
end
end
+284
View File
@@ -0,0 +1,284 @@
module CodeRay
# = PluginHost
#
# A simple subclass/subfolder plugin system.
#
# Example:
# class Generators
# extend PluginHost
# plugin_path 'app/generators'
# end
#
# class Generator
# extend Plugin
# PLUGIN_HOST = Generators
# end
#
# class FancyGenerator < Generator
# register_for :fancy
# end
#
# Generators[:fancy] #-> FancyGenerator
# # or
# CodeRay.require_plugin 'Generators/fancy'
# # or
# Generators::Fancy
module PluginHost
# Raised if Encoders::[] fails because:
# * a file could not be found
# * the requested Plugin is not registered
PluginNotFound = Class.new LoadError
HostNotFound = Class.new LoadError
PLUGIN_HOSTS = []
PLUGIN_HOSTS_BY_ID = {} # dummy hash
# Loads all plugins using list and load.
def load_all
for plugin in list
load plugin
end
end
# Returns the Plugin for +id+.
#
# Example:
# yaml_plugin = MyPluginHost[:yaml]
def [] id, *args, &blk
plugin = validate_id(id)
begin
plugin = plugin_hash.[] plugin, *args, &blk
end while plugin.is_a? Symbol
plugin
end
alias load []
# Tries to +load+ the missing plugin by translating +const+ to the
# underscore form (eg. LinesOfCode becomes lines_of_code).
def const_missing const
id = const.to_s.
gsub(/([A-Z]+)([A-Z][a-z])/,'\1_\2').
gsub(/([a-z\d])([A-Z])/,'\1_\2').
downcase
load id
end
class << self
# Adds the module/class to the PLUGIN_HOSTS list.
def extended mod
PLUGIN_HOSTS << mod
end
end
# The path where the plugins can be found.
def plugin_path *args
unless args.empty?
@plugin_path = File.expand_path File.join(*args)
end
@plugin_path ||= ''
end
# Map a plugin_id to another.
#
# Usage: Put this in a file plugin_path/_map.rb.
#
# class MyColorHost < PluginHost
# map :navy => :dark_blue,
# :maroon => :brown,
# :luna => :moon
# end
def map hash
for from, to in hash
from = validate_id from
to = validate_id to
plugin_hash[from] = to unless plugin_hash.has_key? from
end
end
# Define the default plugin to use when no plugin is found
# for a given id, or return the default plugin.
#
# See also map.
#
# class MyColorHost < PluginHost
# map :navy => :dark_blue
# default :gray
# end
#
# MyColorHost.default # loads and returns the Gray plugin
def default id = nil
if id
id = validate_id id
raise "The default plugin can't be named \"default\"." if id == :default
plugin_hash[:default] = id
else
load :default
end
end
# Every plugin must register itself for +id+ by calling register_for,
# which calls this method.
#
# See Plugin#register_for.
def register plugin, id
plugin_hash[validate_id(id)] = plugin
end
# A Hash of plugion_id => Plugin pairs.
def plugin_hash
@plugin_hash ||= make_plugin_hash
end
# Returns an array of all .rb files in the plugin path.
#
# The extension .rb is not included.
def list
Dir[path_to('*')].select do |file|
File.basename(file)[/^(?!_)\w+\.rb$/]
end.map do |file|
File.basename(file, '.rb').to_sym
end
end
# Returns an array of all Plugins.
#
# Note: This loads all plugins using load_all.
def all_plugins
load_all
plugin_hash.values.grep(Class)
end
# Loads the map file (see map).
#
# This is done automatically when plugin_path is called.
def load_plugin_map
mapfile = path_to '_map'
@plugin_map_loaded = true
if File.exist? mapfile
require mapfile
true
else
false
end
end
protected
# Return a plugin hash that automatically loads plugins.
def make_plugin_hash
@plugin_map_loaded ||= false
Hash.new do |h, plugin_id|
id = validate_id(plugin_id)
path = path_to id
begin
raise LoadError, "#{path} not found" unless File.exist? path
require path
rescue LoadError => boom
if @plugin_map_loaded
if h.has_key?(:default)
warn '%p could not load plugin %p; falling back to %p' % [self, id, h[:default]]
h[:default]
else
raise PluginNotFound, '%p could not load plugin %p: %s' % [self, id, boom]
end
else
load_plugin_map
h[plugin_id]
end
else
# Plugin should have registered by now
if h.has_key? id
h[id]
else
raise PluginNotFound, "No #{self.name} plugin for #{id.inspect} found in #{path}."
end
end
end
end
# Returns the expected path to the plugin file for the given id.
def path_to plugin_id
File.join plugin_path, "#{plugin_id}.rb"
end
# Converts +id+ to a Symbol if it is a String,
# or returns +id+ if it already is a Symbol.
#
# Raises +ArgumentError+ for all other objects, or if the
# given String includes non-alphanumeric characters (\W).
def validate_id id
if id.is_a? Symbol or id.nil?
id
elsif id.is_a? String
if id[/\w+/] == id
id.downcase.to_sym
else
raise ArgumentError, "Invalid id given: #{id}"
end
else
raise ArgumentError, "String or Symbol expected, but #{id.class} given."
end
end
end
# = Plugin
#
# Plugins have to include this module.
#
# IMPORTANT: Use extend for this module.
#
# See CodeRay::PluginHost for examples.
module Plugin
attr_reader :plugin_id
# Register this class for the given +id+.
#
# Example:
# class MyPlugin < PluginHost::BaseClass
# register_for :my_id
# ...
# end
#
# See PluginHost.register.
def register_for id
@plugin_id = id
plugin_host.register self, id
end
# Returns the title of the plugin, or sets it to the
# optional argument +title+.
def title title = nil
if title
@title = title.to_s
else
@title ||= name[/([^:]+)$/, 1]
end
end
# The PluginHost for this Plugin class.
def plugin_host host = nil
if host.is_a? PluginHost
const_set :PLUGIN_HOST, host
end
self::PLUGIN_HOST
end
def aliases
plugin_host.load_plugin_map
plugin_host.plugin_hash.inject [] do |aliases, (key, _)|
aliases << key if plugin_host[key] == self
aliases
end
end
end
end
@@ -0,0 +1,77 @@
module CodeRay
# = WordList
#
# <b>A Hash subclass designed for mapping word lists to token types.</b>
#
# Copyright (c) 2006-2011 by murphy (Kornelius Kalnbach) <murphy rubychan de>
#
# License:: LGPL / ask the author
# Version:: 2.0 (2011-05-08)
#
# A WordList is a Hash with some additional features.
# It is intended to be used for keyword recognition.
#
# WordList is optimized to be used in Scanners,
# typically to decide whether a given ident is a special token.
#
# For case insensitive words use WordList::CaseIgnoring.
#
# Example:
#
# # define word arrays
# RESERVED_WORDS = %w[
# asm break case continue default do else
# ]
#
# PREDEFINED_TYPES = %w[
# int long short char void
# ]
#
# # make a WordList
# IDENT_KIND = WordList.new(:ident).
# add(RESERVED_WORDS, :reserved).
# add(PREDEFINED_TYPES, :predefined_type)
#
# ...
#
# def scan_tokens tokens, options
# ...
#
# elsif scan(/[A-Za-z_][A-Za-z_0-9]*/)
# # use it
# kind = IDENT_KIND[match]
# ...
class WordList < Hash
# Create a new WordList with +default+ as default value.
def initialize default = false
super default
end
# Add words to the list and associate them with +value+.
#
# Returns +self+, so you can concat add calls.
def add words, value = true
words.each { |word| self[word] = value }
self
end
end
# A CaseIgnoring WordList is like a WordList, only that
# keys are compared case-insensitively (normalizing keys using +downcase+).
class WordList::CaseIgnoring < WordList
def [] key
super key.downcase
end
def []= key, value
super key.downcase, value
end
end
end
+323
View File
@@ -0,0 +1,323 @@
# encoding: utf-8
require 'strscan'
module CodeRay
autoload :WordList, 'coderay/helpers/word_list'
# = Scanners
#
# This module holds the Scanner class and its subclasses.
# For example, the Ruby scanner is named CodeRay::Scanners::Ruby
# can be found in coderay/scanners/ruby.
#
# Scanner also provides methods and constants for the register
# mechanism and the [] method that returns the Scanner class
# belonging to the given lang.
#
# See PluginHost.
module Scanners
extend PluginHost
plugin_path File.dirname(__FILE__), 'scanners'
# = Scanner
#
# The base class for all Scanners.
#
# It is a subclass of Ruby's great +StringScanner+, which
# makes it easy to access the scanning methods inside.
#
# It is also +Enumerable+, so you can use it like an Array of
# Tokens:
#
# require 'coderay'
#
# c_scanner = CodeRay::Scanners[:c].new "if (*p == '{') nest++;"
#
# for text, kind in c_scanner
# puts text if kind == :operator
# end
#
# # prints: (*==)++;
#
# OK, this is a very simple example :)
# You can also use +map+, +any?+, +find+ and even +sort_by+,
# if you want.
class Scanner < StringScanner
extend Plugin
plugin_host Scanners
# Raised if a Scanner fails while scanning
ScanError = Class.new StandardError
# The default options for all scanner classes.
#
# Define @default_options for subclasses.
DEFAULT_OPTIONS = { }
KINDS_NOT_LOC = [:comment, :doctype, :docstring]
attr_accessor :state
class << self
# Normalizes the given code into a string with UNIX newlines, in the
# scanner's internal encoding, with invalid and undefined charachters
# replaced by placeholders. Always returns a new object.
def normalize code
# original = code
code = code.to_s unless code.is_a? ::String
return code if code.empty?
if code.respond_to? :encoding
code = encode_with_encoding code, self.encoding
else
code = to_unix code
end
# code = code.dup if code.eql? original
code
end
# The typical filename suffix for this scanner's language.
def file_extension extension = lang
@file_extension ||= extension.to_s
end
# The encoding used internally by this scanner.
def encoding name = 'UTF-8'
@encoding ||= defined?(Encoding.find) && Encoding.find(name)
end
# The lang of this Scanner class, which is equal to its Plugin ID.
def lang
@plugin_id
end
protected
def encode_with_encoding code, target_encoding
if code.encoding == target_encoding
if code.valid_encoding?
return to_unix(code)
else
source_encoding = guess_encoding code
end
else
source_encoding = code.encoding
end
# print "encode_with_encoding from #{source_encoding} to #{target_encoding}"
code.encode target_encoding, source_encoding, :universal_newline => true, :undef => :replace, :invalid => :replace
end
def to_unix code
code.index(?\r) ? code.gsub(/\r\n?/, "\n") : code
end
def guess_encoding s
#:nocov:
IO.popen("file -b --mime -", "w+") do |file|
file.write s[0, 1024]
file.close_write
begin
Encoding.find file.gets[/charset=([-\w]+)/, 1]
rescue ArgumentError
Encoding::BINARY
end
end
#:nocov:
end
end
# Create a new Scanner.
#
# * +code+ is the input String and is handled by the superclass
# StringScanner.
# * +options+ is a Hash with Symbols as keys.
# It is merged with the default options of the class (you can
# overwrite default options here.)
#
# Else, a Tokens object is used.
def initialize code = '', options = {}
if self.class == Scanner
raise NotImplementedError, "I am only the basic Scanner class. I can't scan anything. :( Use my subclasses."
end
@options = self.class::DEFAULT_OPTIONS.merge options
super self.class.normalize(code)
@tokens = options[:tokens] || Tokens.new
@tokens.scanner = self if @tokens.respond_to? :scanner=
setup
end
# Sets back the scanner. Subclasses should redefine the reset_instance
# method instead of this one.
def reset
super
reset_instance
end
# Set a new string to be scanned.
def string= code
code = self.class.normalize(code)
super code
reset_instance
end
# the Plugin ID for this scanner
def lang
self.class.lang
end
# the default file extension for this scanner
def file_extension
self.class.file_extension
end
# Scan the code and returns all tokens in a Tokens object.
def tokenize source = nil, options = {}
options = @options.merge(options)
@tokens = options[:tokens] || @tokens || Tokens.new
@tokens.scanner = self if @tokens.respond_to? :scanner=
case source
when Array
self.string = self.class.normalize(source.join)
when nil
reset
else
self.string = self.class.normalize(source)
end
begin
scan_tokens @tokens, options
rescue => e
message = "Error in %s#scan_tokens, initial state was: %p" % [self.class, defined?(state) && state]
raise_inspect e.message, @tokens, message, 30, e.backtrace
end
@cached_tokens = @tokens
if source.is_a? Array
@tokens.split_into_parts(*source.map { |part| part.size })
else
@tokens
end
end
# Cache the result of tokenize.
def tokens
@cached_tokens ||= tokenize
end
# Traverse the tokens.
def each &block
tokens.each(&block)
end
include Enumerable
# The current line position of the scanner, starting with 1.
# See also: #column.
#
# Beware, this is implemented inefficiently. It should be used
# for debugging only.
def line pos = self.pos
return 1 if pos <= 0
binary_string[0...pos].count("\n") + 1
end
# The current column position of the scanner, starting with 1.
# See also: #line.
def column pos = self.pos
return 1 if pos <= 0
pos - (binary_string.rindex(?\n, pos - 1) || -1)
end
# The string in binary encoding.
#
# To be used with #pos, which is the index of the byte the scanner
# will scan next.
def binary_string
@binary_string ||=
if string.respond_to?(:bytesize) && string.bytesize != string.size
#:nocov:
string.dup.force_encoding('binary')
#:nocov:
else
string
end
end
protected
# Can be implemented by subclasses to do some initialization
# that has to be done once per instance.
#
# Use reset for initialization that has to be done once per
# scan.
def setup # :doc:
end
# This is the central method, and commonly the only one a
# subclass implements.
#
# Subclasses must implement this method; it must return +tokens+
# and must only use Tokens#<< for storing scanned tokens!
def scan_tokens tokens, options # :doc:
raise NotImplementedError, "#{self.class}#scan_tokens not implemented."
end
# Resets the scanner.
def reset_instance
@tokens.clear if @tokens.respond_to?(:clear) && !@options[:keep_tokens]
@cached_tokens = nil
@binary_string = nil if defined? @binary_string
end
# Scanner error with additional status information
def raise_inspect msg, tokens, state = self.state || 'No state given!', ambit = 30, backtrace = caller
raise ScanError, <<-EOE % [
***ERROR in %s: %s (after %d tokens)
tokens:
%s
current line: %d column: %d pos: %d
matched: %p state: %p
bol? = %p, eos? = %p
surrounding code:
%p ~~ %p
***ERROR***
EOE
File.basename(caller[0]),
msg,
tokens.respond_to?(:size) ? tokens.size : 0,
tokens.respond_to?(:last) ? tokens.last(10).map { |t| t.inspect }.join("\n") : '',
line, column, pos,
matched, state, bol?, eos?,
binary_string[pos - ambit, ambit],
binary_string[pos, ambit],
], backtrace
end
# Shorthand for scan_until(/\z/).
# This method also avoids a JRuby 1.9 mode bug.
def scan_rest
rest = self.rest
terminate
rest
end
end
end
end
@@ -0,0 +1,421 @@
module CodeRay
module Scanners
module Java::BuiltinTypes # :nodoc:
#:nocov:
List = %w[
AbstractAction AbstractBorder AbstractButton AbstractCellEditor AbstractCollection
AbstractColorChooserPanel AbstractDocument AbstractExecutorService AbstractInterruptibleChannel
AbstractLayoutCache AbstractList AbstractListModel AbstractMap AbstractMethodError AbstractPreferences
AbstractQueue AbstractQueuedSynchronizer AbstractSelectableChannel AbstractSelectionKey AbstractSelector
AbstractSequentialList AbstractSet AbstractSpinnerModel AbstractTableModel AbstractUndoableEdit
AbstractWriter AccessControlContext AccessControlException AccessController AccessException Accessible
AccessibleAction AccessibleAttributeSequence AccessibleBundle AccessibleComponent AccessibleContext
AccessibleEditableText AccessibleExtendedComponent AccessibleExtendedTable AccessibleExtendedText
AccessibleHyperlink AccessibleHypertext AccessibleIcon AccessibleKeyBinding AccessibleObject
AccessibleRelation AccessibleRelationSet AccessibleResourceBundle AccessibleRole AccessibleSelection
AccessibleState AccessibleStateSet AccessibleStreamable AccessibleTable AccessibleTableModelChange
AccessibleText AccessibleTextSequence AccessibleValue AccountException AccountExpiredException
AccountLockedException AccountNotFoundException Acl AclEntry AclNotFoundException Action ActionEvent
ActionListener ActionMap ActionMapUIResource Activatable ActivateFailedException ActivationDesc
ActivationException ActivationGroup ActivationGroupDesc ActivationGroupID ActivationGroup_Stub
ActivationID ActivationInstantiator ActivationMonitor ActivationSystem Activator ActiveEvent
ActivityCompletedException ActivityRequiredException Adjustable AdjustmentEvent AdjustmentListener
Adler32 AffineTransform AffineTransformOp AlgorithmParameterGenerator AlgorithmParameterGeneratorSpi
AlgorithmParameters AlgorithmParameterSpec AlgorithmParametersSpi AllPermission AlphaComposite
AlreadyBoundException AlreadyConnectedException AncestorEvent AncestorListener AnnotatedElement
Annotation AnnotationFormatError AnnotationTypeMismatchException AppConfigurationEntry Appendable Applet
AppletContext AppletInitializer AppletStub Arc2D Area AreaAveragingScaleFilter ArithmeticException Array
ArrayBlockingQueue ArrayIndexOutOfBoundsException ArrayList Arrays ArrayStoreException ArrayType
AssertionError AsyncBoxView AsynchronousCloseException AtomicBoolean AtomicInteger AtomicIntegerArray
AtomicIntegerFieldUpdater AtomicLong AtomicLongArray AtomicLongFieldUpdater AtomicMarkableReference
AtomicReference AtomicReferenceArray AtomicReferenceFieldUpdater AtomicStampedReference Attribute
AttributeChangeNotification AttributeChangeNotificationFilter AttributedCharacterIterator
AttributedString AttributeException AttributeInUseException AttributeList AttributeModificationException
AttributeNotFoundException Attributes AttributeSet AttributeSetUtilities AttributeValueExp AudioClip
AudioFileFormat AudioFileReader AudioFileWriter AudioFormat AudioInputStream AudioPermission AudioSystem
AuthenticationException AuthenticationNotSupportedException Authenticator AuthorizeCallback
AuthPermission AuthProvider Autoscroll AWTError AWTEvent AWTEventListener AWTEventListenerProxy
AWTEventMulticaster AWTException AWTKeyStroke AWTPermission BackingStoreException
BadAttributeValueExpException BadBinaryOpValueExpException BadLocationException BadPaddingException
BadStringOperationException BandCombineOp BandedSampleModel BaseRowSet BasicArrowButton BasicAttribute
BasicAttributes BasicBorders BasicButtonListener BasicButtonUI BasicCheckBoxMenuItemUI BasicCheckBoxUI
BasicColorChooserUI BasicComboBoxEditor BasicComboBoxRenderer BasicComboBoxUI BasicComboPopup
BasicControl BasicDesktopIconUI BasicDesktopPaneUI BasicDirectoryModel BasicEditorPaneUI
BasicFileChooserUI BasicFormattedTextFieldUI BasicGraphicsUtils BasicHTML BasicIconFactory
BasicInternalFrameTitlePane BasicInternalFrameUI BasicLabelUI BasicListUI BasicLookAndFeel
BasicMenuBarUI BasicMenuItemUI BasicMenuUI BasicOptionPaneUI BasicPanelUI BasicPasswordFieldUI
BasicPermission BasicPopupMenuSeparatorUI BasicPopupMenuUI BasicProgressBarUI BasicRadioButtonMenuItemUI
BasicRadioButtonUI BasicRootPaneUI BasicScrollBarUI BasicScrollPaneUI BasicSeparatorUI BasicSliderUI
BasicSpinnerUI BasicSplitPaneDivider BasicSplitPaneUI BasicStroke BasicTabbedPaneUI BasicTableHeaderUI
BasicTableUI BasicTextAreaUI BasicTextFieldUI BasicTextPaneUI BasicTextUI BasicToggleButtonUI
BasicToolBarSeparatorUI BasicToolBarUI BasicToolTipUI BasicTreeUI BasicViewportUI BatchUpdateException
BeanContext BeanContextChild BeanContextChildComponentProxy BeanContextChildSupport
BeanContextContainerProxy BeanContextEvent BeanContextMembershipEvent BeanContextMembershipListener
BeanContextProxy BeanContextServiceAvailableEvent BeanContextServiceProvider
BeanContextServiceProviderBeanInfo BeanContextServiceRevokedEvent BeanContextServiceRevokedListener
BeanContextServices BeanContextServicesListener BeanContextServicesSupport BeanContextSupport
BeanDescriptor BeanInfo Beans BevelBorder Bidi BigDecimal BigInteger BinaryRefAddr BindException Binding
BitSet Blob BlockingQueue BlockView BMPImageWriteParam Book Boolean BooleanControl Border BorderFactory
BorderLayout BorderUIResource BoundedRangeModel Box BoxLayout BoxView BreakIterator
BrokenBarrierException Buffer BufferCapabilities BufferedImage BufferedImageFilter BufferedImageOp
BufferedInputStream BufferedOutputStream BufferedReader BufferedWriter BufferOverflowException
BufferStrategy BufferUnderflowException Button ButtonGroup ButtonModel ButtonUI Byte
ByteArrayInputStream ByteArrayOutputStream ByteBuffer ByteChannel ByteLookupTable ByteOrder CachedRowSet
CacheRequest CacheResponse Calendar Callable CallableStatement Callback CallbackHandler
CancelablePrintJob CancellationException CancelledKeyException CannotProceedException
CannotRedoException CannotUndoException Canvas CardLayout Caret CaretEvent CaretListener CellEditor
CellEditorListener CellRendererPane Certificate CertificateEncodingException CertificateException
CertificateExpiredException CertificateFactory CertificateFactorySpi CertificateNotYetValidException
CertificateParsingException CertPath CertPathBuilder CertPathBuilderException CertPathBuilderResult
CertPathBuilderSpi CertPathParameters CertPathTrustManagerParameters CertPathValidator
CertPathValidatorException CertPathValidatorResult CertPathValidatorSpi CertSelector CertStore
CertStoreException CertStoreParameters CertStoreSpi ChangedCharSetException ChangeEvent ChangeListener
Channel Channels Character CharacterCodingException CharacterIterator CharArrayReader CharArrayWriter
CharBuffer CharConversionException CharSequence Charset CharsetDecoder CharsetEncoder CharsetProvider
Checkbox CheckboxGroup CheckboxMenuItem CheckedInputStream CheckedOutputStream Checksum Choice
ChoiceCallback ChoiceFormat Chromaticity Cipher CipherInputStream CipherOutputStream CipherSpi Class
ClassCastException ClassCircularityError ClassDefinition ClassDesc ClassFileTransformer ClassFormatError
ClassLoader ClassLoaderRepository ClassLoadingMXBean ClassNotFoundException Clip Clipboard
ClipboardOwner Clob Cloneable CloneNotSupportedException Closeable ClosedByInterruptException
ClosedChannelException ClosedSelectorException CMMException CoderMalfunctionError CoderResult CodeSigner
CodeSource CodingErrorAction CollationElementIterator CollationKey Collator Collection
CollectionCertStoreParameters Collections Color ColorChooserComponentFactory ColorChooserUI
ColorConvertOp ColorModel ColorSelectionModel ColorSpace ColorSupported ColorType ColorUIResource
ComboBoxEditor ComboBoxModel ComboBoxUI ComboPopup CommunicationException Comparable Comparator
CompilationMXBean Compiler CompletionService Component ComponentAdapter ComponentColorModel
ComponentEvent ComponentInputMap ComponentInputMapUIResource ComponentListener ComponentOrientation
ComponentSampleModel ComponentUI ComponentView Composite CompositeContext CompositeData
CompositeDataSupport CompositeName CompositeType CompositeView CompoundBorder CompoundControl
CompoundEdit CompoundName Compression ConcurrentHashMap ConcurrentLinkedQueue ConcurrentMap
ConcurrentModificationException Condition Configuration ConfigurationException ConfirmationCallback
ConnectException ConnectIOException Connection ConnectionEvent ConnectionEventListener
ConnectionPendingException ConnectionPoolDataSource ConsoleHandler Constructor Container
ContainerAdapter ContainerEvent ContainerListener ContainerOrderFocusTraversalPolicy ContentHandler
ContentHandlerFactory ContentModel Context ContextNotEmptyException ContextualRenderedImageFactory
Control ControlFactory ControllerEventListener ConvolveOp CookieHandler Copies CopiesSupported
CopyOnWriteArrayList CopyOnWriteArraySet CountDownLatch CounterMonitor CounterMonitorMBean CRC32
CredentialException CredentialExpiredException CredentialNotFoundException CRL CRLException CRLSelector
CropImageFilter CSS CubicCurve2D Currency Cursor Customizer CyclicBarrier DatabaseMetaData DataBuffer
DataBufferByte DataBufferDouble DataBufferFloat DataBufferInt DataBufferShort DataBufferUShort
DataFlavor DataFormatException DatagramChannel DatagramPacket DatagramSocket DatagramSocketImpl
DatagramSocketImplFactory DataInput DataInputStream DataLine DataOutput DataOutputStream DataSource
DataTruncation DatatypeConfigurationException DatatypeConstants DatatypeFactory Date DateFormat
DateFormatSymbols DateFormatter DateTimeAtCompleted DateTimeAtCreation DateTimeAtProcessing
DateTimeSyntax DebugGraphics DecimalFormat DecimalFormatSymbols DefaultBoundedRangeModel
DefaultButtonModel DefaultCaret DefaultCellEditor DefaultColorSelectionModel DefaultComboBoxModel
DefaultDesktopManager DefaultEditorKit DefaultFocusManager DefaultFocusTraversalPolicy DefaultFormatter
DefaultFormatterFactory DefaultHighlighter DefaultKeyboardFocusManager DefaultListCellRenderer
DefaultListModel DefaultListSelectionModel DefaultLoaderRepository DefaultMenuLayout DefaultMetalTheme
DefaultMutableTreeNode DefaultPersistenceDelegate DefaultSingleSelectionModel DefaultStyledDocument
DefaultTableCellRenderer DefaultTableColumnModel DefaultTableModel DefaultTextUI DefaultTreeCellEditor
DefaultTreeCellRenderer DefaultTreeModel DefaultTreeSelectionModel Deflater DeflaterOutputStream Delayed
DelayQueue DelegationPermission Deprecated Descriptor DescriptorAccess DescriptorSupport DESedeKeySpec
DesignMode DESKeySpec DesktopIconUI DesktopManager DesktopPaneUI Destination Destroyable
DestroyFailedException DGC DHGenParameterSpec DHKey DHParameterSpec DHPrivateKey DHPrivateKeySpec
DHPublicKey DHPublicKeySpec Dialog Dictionary DigestException DigestInputStream DigestOutputStream
Dimension Dimension2D DimensionUIResource DirContext DirectColorModel DirectoryManager DirObjectFactory
DirStateFactory DisplayMode DnDConstants Doc DocAttribute DocAttributeSet DocFlavor DocPrintJob Document
DocumentBuilder DocumentBuilderFactory Documented DocumentEvent DocumentFilter DocumentListener
DocumentName DocumentParser DomainCombiner DOMLocator DOMResult DOMSource Double DoubleBuffer
DragGestureEvent DragGestureListener DragGestureRecognizer DragSource DragSourceAdapter
DragSourceContext DragSourceDragEvent DragSourceDropEvent DragSourceEvent DragSourceListener
DragSourceMotionListener Driver DriverManager DriverPropertyInfo DropTarget DropTargetAdapter
DropTargetContext DropTargetDragEvent DropTargetDropEvent DropTargetEvent DropTargetListener DSAKey
DSAKeyPairGenerator DSAParameterSpec DSAParams DSAPrivateKey DSAPrivateKeySpec DSAPublicKey
DSAPublicKeySpec DTD DTDConstants DuplicateFormatFlagsException Duration DynamicMBean ECField ECFieldF2m
ECFieldFp ECGenParameterSpec ECKey ECParameterSpec ECPoint ECPrivateKey ECPrivateKeySpec ECPublicKey
ECPublicKeySpec EditorKit Element ElementIterator ElementType Ellipse2D EllipticCurve EmptyBorder
EmptyStackException EncodedKeySpec Encoder EncryptedPrivateKeyInfo Entity Enum
EnumConstantNotPresentException EnumControl Enumeration EnumMap EnumSet EnumSyntax EOFException Error
ErrorListener ErrorManager EtchedBorder Event EventContext EventDirContext EventHandler EventListener
EventListenerList EventListenerProxy EventObject EventQueue EventSetDescriptor Exception
ExceptionInInitializerError ExceptionListener Exchanger ExecutionException Executor
ExecutorCompletionService Executors ExecutorService ExemptionMechanism ExemptionMechanismException
ExemptionMechanismSpi ExpandVetoException ExportException Expression ExtendedRequest ExtendedResponse
Externalizable FactoryConfigurationError FailedLoginException FeatureDescriptor Fidelity Field
FieldPosition FieldView File FileCacheImageInputStream FileCacheImageOutputStream FileChannel
FileChooserUI FileDescriptor FileDialog FileFilter FileHandler FileImageInputStream
FileImageOutputStream FileInputStream FileLock FileLockInterruptionException FilenameFilter FileNameMap
FileNotFoundException FileOutputStream FilePermission FileReader FileSystemView FileView FileWriter
Filter FilteredImageSource FilteredRowSet FilterInputStream FilterOutputStream FilterReader FilterWriter
Finishings FixedHeightLayoutCache FlatteningPathIterator FlavorEvent FlavorException FlavorListener
FlavorMap FlavorTable Float FloatBuffer FloatControl FlowLayout FlowView Flushable FocusAdapter
FocusEvent FocusListener FocusManager FocusTraversalPolicy Font FontFormatException FontMetrics
FontRenderContext FontUIResource Format FormatConversionProvider FormatFlagsConversionMismatchException
Formattable FormattableFlags Formatter FormatterClosedException FormSubmitEvent FormView Frame Future
FutureTask GapContent GarbageCollectorMXBean GatheringByteChannel GaugeMonitor GaugeMonitorMBean
GeneralPath GeneralSecurityException GenericArrayType GenericDeclaration GenericSignatureFormatError
GlyphJustificationInfo GlyphMetrics GlyphVector GlyphView GradientPaint GraphicAttribute Graphics
Graphics2D GraphicsConfigTemplate GraphicsConfiguration GraphicsDevice GraphicsEnvironment GrayFilter
GregorianCalendar GridBagConstraints GridBagLayout GridLayout Group Guard GuardedObject GZIPInputStream
GZIPOutputStream Handler HandshakeCompletedEvent HandshakeCompletedListener HasControls HashAttributeSet
HashDocAttributeSet HashMap HashPrintJobAttributeSet HashPrintRequestAttributeSet
HashPrintServiceAttributeSet HashSet Hashtable HeadlessException HierarchyBoundsAdapter
HierarchyBoundsListener HierarchyEvent HierarchyListener Highlighter HostnameVerifier HTML HTMLDocument
HTMLEditorKit HTMLFrameHyperlinkEvent HTMLWriter HttpRetryException HttpsURLConnection HttpURLConnection
HyperlinkEvent HyperlinkListener ICC_ColorSpace ICC_Profile ICC_ProfileGray ICC_ProfileRGB Icon
IconUIResource IconView Identity IdentityHashMap IdentityScope IIOByteBuffer IIOException IIOImage
IIOInvalidTreeException IIOMetadata IIOMetadataController IIOMetadataFormat IIOMetadataFormatImpl
IIOMetadataNode IIOParam IIOParamController IIOReadProgressListener IIOReadUpdateListener
IIOReadWarningListener IIORegistry IIOServiceProvider IIOWriteProgressListener IIOWriteWarningListener
IllegalAccessError IllegalAccessException IllegalArgumentException IllegalBlockingModeException
IllegalBlockSizeException IllegalCharsetNameException IllegalClassFormatException
IllegalComponentStateException IllegalFormatCodePointException IllegalFormatConversionException
IllegalFormatException IllegalFormatFlagsException IllegalFormatPrecisionException
IllegalFormatWidthException IllegalMonitorStateException IllegalPathStateException
IllegalSelectorException IllegalStateException IllegalThreadStateException Image ImageCapabilities
ImageConsumer ImageFilter ImageGraphicAttribute ImageIcon ImageInputStream ImageInputStreamImpl
ImageInputStreamSpi ImageIO ImageObserver ImageOutputStream ImageOutputStreamImpl ImageOutputStreamSpi
ImageProducer ImageReader ImageReaderSpi ImageReaderWriterSpi ImageReadParam ImageTranscoder
ImageTranscoderSpi ImageTypeSpecifier ImageView ImageWriteParam ImageWriter ImageWriterSpi
ImagingOpException IncompatibleClassChangeError IncompleteAnnotationException IndexColorModel
IndexedPropertyChangeEvent IndexedPropertyDescriptor IndexOutOfBoundsException Inet4Address Inet6Address
InetAddress InetSocketAddress Inflater InflaterInputStream InheritableThreadLocal Inherited
InitialContext InitialContextFactory InitialContextFactoryBuilder InitialDirContext InitialLdapContext
InlineView InputContext InputEvent InputMap InputMapUIResource InputMethod InputMethodContext
InputMethodDescriptor InputMethodEvent InputMethodHighlight InputMethodListener InputMethodRequests
InputMismatchException InputStream InputStreamReader InputSubset InputVerifier Insets InsetsUIResource
InstanceAlreadyExistsException InstanceNotFoundException InstantiationError InstantiationException
Instrument Instrumentation InsufficientResourcesException IntBuffer Integer IntegerSyntax InternalError
InternalFrameAdapter InternalFrameEvent InternalFrameFocusTraversalPolicy InternalFrameListener
InternalFrameUI InternationalFormatter InterruptedException InterruptedIOException
InterruptedNamingException InterruptibleChannel IntrospectionException Introspector
InvalidActivityException InvalidAlgorithmParameterException InvalidApplicationException
InvalidAttributeIdentifierException InvalidAttributesException InvalidAttributeValueException
InvalidClassException InvalidDnDOperationException InvalidKeyException InvalidKeySpecException
InvalidMarkException InvalidMidiDataException InvalidNameException InvalidObjectException
InvalidOpenTypeException InvalidParameterException InvalidParameterSpecException
InvalidPreferencesFormatException InvalidPropertiesFormatException InvalidRelationIdException
InvalidRelationServiceException InvalidRelationTypeException InvalidRoleInfoException
InvalidRoleValueException InvalidSearchControlsException InvalidSearchFilterException
InvalidTargetObjectTypeException InvalidTransactionException InvocationEvent InvocationHandler
InvocationTargetException IOException ItemEvent ItemListener ItemSelectable Iterable Iterator
IvParameterSpec JApplet JarEntry JarException JarFile JarInputStream JarOutputStream JarURLConnection
JButton JCheckBox JCheckBoxMenuItem JColorChooser JComboBox JComponent JdbcRowSet JDesktopPane JDialog
JEditorPane JFileChooser JFormattedTextField JFrame JInternalFrame JLabel JLayeredPane JList JMenu
JMenuBar JMenuItem JMException JMRuntimeException JMXAuthenticator JMXConnectionNotification
JMXConnector JMXConnectorFactory JMXConnectorProvider JMXConnectorServer JMXConnectorServerFactory
JMXConnectorServerMBean JMXConnectorServerProvider JMXPrincipal JMXProviderException
JMXServerErrorException JMXServiceURL JobAttributes JobHoldUntil JobImpressions JobImpressionsCompleted
JobImpressionsSupported JobKOctets JobKOctetsProcessed JobKOctetsSupported JobMediaSheets
JobMediaSheetsCompleted JobMediaSheetsSupported JobMessageFromOperator JobName JobOriginatingUserName
JobPriority JobPrioritySupported JobSheets JobState JobStateReason JobStateReasons Joinable JoinRowSet
JOptionPane JPanel JPasswordField JPEGHuffmanTable JPEGImageReadParam JPEGImageWriteParam JPEGQTable
JPopupMenu JProgressBar JRadioButton JRadioButtonMenuItem JRootPane JScrollBar JScrollPane JSeparator
JSlider JSpinner JSplitPane JTabbedPane JTable JTableHeader JTextArea JTextComponent JTextField
JTextPane JToggleButton JToolBar JToolTip JTree JViewport JWindow KerberosKey KerberosPrincipal
KerberosTicket Kernel Key KeyAdapter KeyAgreement KeyAgreementSpi KeyAlreadyExistsException
KeyboardFocusManager KeyEvent KeyEventDispatcher KeyEventPostProcessor KeyException KeyFactory
KeyFactorySpi KeyGenerator KeyGeneratorSpi KeyListener KeyManagementException KeyManager
KeyManagerFactory KeyManagerFactorySpi Keymap KeyPair KeyPairGenerator KeyPairGeneratorSpi KeyRep
KeySpec KeyStore KeyStoreBuilderParameters KeyStoreException KeyStoreSpi KeyStroke Label LabelUI
LabelView LanguageCallback LastOwnerException LayeredHighlighter LayoutFocusTraversalPolicy
LayoutManager LayoutManager2 LayoutQueue LDAPCertStoreParameters LdapContext LdapName
LdapReferralException Lease Level LimitExceededException Line Line2D LineBorder LineBreakMeasurer
LineEvent LineListener LineMetrics LineNumberInputStream LineNumberReader LineUnavailableException
LinkageError LinkedBlockingQueue LinkedHashMap LinkedHashSet LinkedList LinkException LinkLoopException
LinkRef List ListCellRenderer ListDataEvent ListDataListener ListenerNotFoundException ListIterator
ListModel ListResourceBundle ListSelectionEvent ListSelectionListener ListSelectionModel ListUI ListView
LoaderHandler Locale LocateRegistry Lock LockSupport Logger LoggingMXBean LoggingPermission LoginContext
LoginException LoginModule LogManager LogRecord LogStream Long LongBuffer LookAndFeel LookupOp
LookupTable Mac MacSpi MalformedInputException MalformedLinkException MalformedObjectNameException
MalformedParameterizedTypeException MalformedURLException ManagementFactory ManagementPermission
ManageReferralControl ManagerFactoryParameters Manifest Map MappedByteBuffer MarshalException
MarshalledObject MaskFormatter Matcher MatchResult Math MathContext MatteBorder MBeanAttributeInfo
MBeanConstructorInfo MBeanException MBeanFeatureInfo MBeanInfo MBeanNotificationInfo MBeanOperationInfo
MBeanParameterInfo MBeanPermission MBeanRegistration MBeanRegistrationException MBeanServer
MBeanServerBuilder MBeanServerConnection MBeanServerDelegate MBeanServerDelegateMBean MBeanServerFactory
MBeanServerForwarder MBeanServerInvocationHandler MBeanServerNotification MBeanServerNotificationFilter
MBeanServerPermission MBeanTrustPermission Media MediaName MediaPrintableArea MediaSize MediaSizeName
MediaTracker MediaTray Member MemoryCacheImageInputStream MemoryCacheImageOutputStream MemoryHandler
MemoryImageSource MemoryManagerMXBean MemoryMXBean MemoryNotificationInfo MemoryPoolMXBean MemoryType
MemoryUsage Menu MenuBar MenuBarUI MenuComponent MenuContainer MenuDragMouseEvent MenuDragMouseListener
MenuElement MenuEvent MenuItem MenuItemUI MenuKeyEvent MenuKeyListener MenuListener MenuSelectionManager
MenuShortcut MessageDigest MessageDigestSpi MessageFormat MetaEventListener MetalBorders MetalButtonUI
MetalCheckBoxIcon MetalCheckBoxUI MetalComboBoxButton MetalComboBoxEditor MetalComboBoxIcon
MetalComboBoxUI MetalDesktopIconUI MetalFileChooserUI MetalIconFactory MetalInternalFrameTitlePane
MetalInternalFrameUI MetalLabelUI MetalLookAndFeel MetalMenuBarUI MetalPopupMenuSeparatorUI
MetalProgressBarUI MetalRadioButtonUI MetalRootPaneUI MetalScrollBarUI MetalScrollButton
MetalScrollPaneUI MetalSeparatorUI MetalSliderUI MetalSplitPaneUI MetalTabbedPaneUI MetalTextFieldUI
MetalTheme MetalToggleButtonUI MetalToolBarUI MetalToolTipUI MetalTreeUI MetaMessage Method
MethodDescriptor MGF1ParameterSpec MidiChannel MidiDevice MidiDeviceProvider MidiEvent MidiFileFormat
MidiFileReader MidiFileWriter MidiMessage MidiSystem MidiUnavailableException MimeTypeParseException
MinimalHTMLWriter MissingFormatArgumentException MissingFormatWidthException MissingResourceException
Mixer MixerProvider MLet MLetMBean ModelMBean ModelMBeanAttributeInfo ModelMBeanConstructorInfo
ModelMBeanInfo ModelMBeanInfoSupport ModelMBeanNotificationBroadcaster ModelMBeanNotificationInfo
ModelMBeanOperationInfo ModificationItem Modifier Monitor MonitorMBean MonitorNotification
MonitorSettingException MouseAdapter MouseDragGestureRecognizer MouseEvent MouseInfo MouseInputAdapter
MouseInputListener MouseListener MouseMotionAdapter MouseMotionListener MouseWheelEvent
MouseWheelListener MultiButtonUI MulticastSocket MultiColorChooserUI MultiComboBoxUI MultiDesktopIconUI
MultiDesktopPaneUI MultiDoc MultiDocPrintJob MultiDocPrintService MultiFileChooserUI
MultiInternalFrameUI MultiLabelUI MultiListUI MultiLookAndFeel MultiMenuBarUI MultiMenuItemUI
MultiOptionPaneUI MultiPanelUI MultiPixelPackedSampleModel MultipleDocumentHandling MultipleMaster
MultiPopupMenuUI MultiProgressBarUI MultiRootPaneUI MultiScrollBarUI MultiScrollPaneUI MultiSeparatorUI
MultiSliderUI MultiSpinnerUI MultiSplitPaneUI MultiTabbedPaneUI MultiTableHeaderUI MultiTableUI
MultiTextUI MultiToolBarUI MultiToolTipUI MultiTreeUI MultiViewportUI MutableAttributeSet
MutableComboBoxModel MutableTreeNode Name NameAlreadyBoundException NameCallback NameClassPair
NameNotFoundException NameParser NamespaceChangeListener NamespaceContext Naming NamingEnumeration
NamingEvent NamingException NamingExceptionEvent NamingListener NamingManager NamingSecurityException
NavigationFilter NegativeArraySizeException NetPermission NetworkInterface NoClassDefFoundError
NoConnectionPendingException NodeChangeEvent NodeChangeListener NoInitialContextException
NoninvertibleTransformException NonReadableChannelException NonWritableChannelException
NoPermissionException NoRouteToHostException NoSuchAlgorithmException NoSuchAttributeException
NoSuchElementException NoSuchFieldError NoSuchFieldException NoSuchMethodError NoSuchMethodException
NoSuchObjectException NoSuchPaddingException NoSuchProviderException NotActiveException
NotBoundException NotCompliantMBeanException NotContextException Notification NotificationBroadcaster
NotificationBroadcasterSupport NotificationEmitter NotificationFilter NotificationFilterSupport
NotificationListener NotificationResult NotOwnerException NotSerializableException NotYetBoundException
NotYetConnectedException NullCipher NullPointerException Number NumberFormat NumberFormatException
NumberFormatter NumberOfDocuments NumberOfInterveningJobs NumberUp NumberUpSupported NumericShaper
OAEPParameterSpec Object ObjectChangeListener ObjectFactory ObjectFactoryBuilder ObjectInput
ObjectInputStream ObjectInputValidation ObjectInstance ObjectName ObjectOutput ObjectOutputStream
ObjectStreamClass ObjectStreamConstants ObjectStreamException ObjectStreamField ObjectView ObjID
Observable Observer OceanTheme OpenDataException OpenMBeanAttributeInfo OpenMBeanAttributeInfoSupport
OpenMBeanConstructorInfo OpenMBeanConstructorInfoSupport OpenMBeanInfo OpenMBeanInfoSupport
OpenMBeanOperationInfo OpenMBeanOperationInfoSupport OpenMBeanParameterInfo
OpenMBeanParameterInfoSupport OpenType OperatingSystemMXBean Operation OperationNotSupportedException
OperationsException Option OptionalDataException OptionPaneUI OrientationRequested OutOfMemoryError
OutputDeviceAssigned OutputKeys OutputStream OutputStreamWriter OverlappingFileLockException
OverlayLayout Override Owner Pack200 Package PackedColorModel Pageable PageAttributes
PagedResultsControl PagedResultsResponseControl PageFormat PageRanges PagesPerMinute PagesPerMinuteColor
Paint PaintContext PaintEvent Panel PanelUI Paper ParagraphView ParameterBlock ParameterDescriptor
ParameterizedType ParameterMetaData ParseException ParsePosition Parser ParserConfigurationException
ParserDelegator PartialResultException PasswordAuthentication PasswordCallback PasswordView Patch
PathIterator Pattern PatternSyntaxException PBEKey PBEKeySpec PBEParameterSpec PDLOverrideSupported
Permission PermissionCollection Permissions PersistenceDelegate PersistentMBean PhantomReference Pipe
PipedInputStream PipedOutputStream PipedReader PipedWriter PixelGrabber PixelInterleavedSampleModel
PKCS8EncodedKeySpec PKIXBuilderParameters PKIXCertPathBuilderResult PKIXCertPathChecker
PKIXCertPathValidatorResult PKIXParameters PlainDocument PlainView Point Point2D PointerInfo Policy
PolicyNode PolicyQualifierInfo Polygon PooledConnection Popup PopupFactory PopupMenu PopupMenuEvent
PopupMenuListener PopupMenuUI Port PortableRemoteObject PortableRemoteObjectDelegate
PortUnreachableException Position Predicate PreferenceChangeEvent PreferenceChangeListener Preferences
PreferencesFactory PreparedStatement PresentationDirection Principal Printable PrinterAbortException
PrinterException PrinterGraphics PrinterInfo PrinterIOException PrinterIsAcceptingJobs PrinterJob
PrinterLocation PrinterMakeAndModel PrinterMessageFromOperator PrinterMoreInfo
PrinterMoreInfoManufacturer PrinterName PrinterResolution PrinterState PrinterStateReason
PrinterStateReasons PrinterURI PrintEvent PrintException PrintGraphics PrintJob PrintJobAdapter
PrintJobAttribute PrintJobAttributeEvent PrintJobAttributeListener PrintJobAttributeSet PrintJobEvent
PrintJobListener PrintQuality PrintRequestAttribute PrintRequestAttributeSet PrintService
PrintServiceAttribute PrintServiceAttributeEvent PrintServiceAttributeListener PrintServiceAttributeSet
PrintServiceLookup PrintStream PrintWriter PriorityBlockingQueue PriorityQueue PrivateClassLoader
PrivateCredentialPermission PrivateKey PrivateMLet PrivilegedAction PrivilegedActionException
PrivilegedExceptionAction Process ProcessBuilder ProfileDataException ProgressBarUI ProgressMonitor
ProgressMonitorInputStream Properties PropertyChangeEvent PropertyChangeListener
PropertyChangeListenerProxy PropertyChangeSupport PropertyDescriptor PropertyEditor
PropertyEditorManager PropertyEditorSupport PropertyPermission PropertyResourceBundle
PropertyVetoException ProtectionDomain ProtocolException Provider ProviderException Proxy ProxySelector
PSource PSSParameterSpec PublicKey PushbackInputStream PushbackReader QName QuadCurve2D Query QueryEval
QueryExp Queue QueuedJobCount Random RandomAccess RandomAccessFile Raster RasterFormatException RasterOp
RC2ParameterSpec RC5ParameterSpec Rdn Readable ReadableByteChannel Reader ReadOnlyBufferException
ReadWriteLock RealmCallback RealmChoiceCallback Receiver Rectangle Rectangle2D RectangularShape
ReentrantLock ReentrantReadWriteLock Ref RefAddr Reference Referenceable ReferenceQueue
ReferenceUriSchemesSupported ReferralException ReflectionException ReflectPermission Refreshable
RefreshFailedException Region RegisterableService Registry RegistryHandler RejectedExecutionException
RejectedExecutionHandler Relation RelationException RelationNotFoundException RelationNotification
RelationService RelationServiceMBean RelationServiceNotRegisteredException RelationSupport
RelationSupportMBean RelationType RelationTypeNotFoundException RelationTypeSupport Remote RemoteCall
RemoteException RemoteObject RemoteObjectInvocationHandler RemoteRef RemoteServer RemoteStub
RenderableImage RenderableImageOp RenderableImageProducer RenderContext RenderedImage
RenderedImageFactory Renderer RenderingHints RepaintManager ReplicateScaleFilter RequestingUserName
RequiredModelMBean RescaleOp ResolutionSyntax Resolver ResolveResult ResourceBundle ResponseCache Result
ResultSet ResultSetMetaData Retention RetentionPolicy ReverbType RGBImageFilter RMIClassLoader
RMIClassLoaderSpi RMIClientSocketFactory RMIConnection RMIConnectionImpl RMIConnectionImpl_Stub
RMIConnector RMIConnectorServer RMIFailureHandler RMIIIOPServerImpl RMIJRMPServerImpl
RMISecurityException RMISecurityManager RMIServer RMIServerImpl RMIServerImpl_Stub
RMIServerSocketFactory RMISocketFactory Robot Role RoleInfo RoleInfoNotFoundException RoleList
RoleNotFoundException RoleResult RoleStatus RoleUnresolved RoleUnresolvedList RootPaneContainer
RootPaneUI RoundingMode RoundRectangle2D RowMapper RowSet RowSetEvent RowSetInternal RowSetListener
RowSetMetaData RowSetMetaDataImpl RowSetReader RowSetWarning RowSetWriter RSAKey RSAKeyGenParameterSpec
RSAMultiPrimePrivateCrtKey RSAMultiPrimePrivateCrtKeySpec RSAOtherPrimeInfo RSAPrivateCrtKey
RSAPrivateCrtKeySpec RSAPrivateKey RSAPrivateKeySpec RSAPublicKey RSAPublicKeySpec RTFEditorKit
RuleBasedCollator Runnable Runtime RuntimeErrorException RuntimeException RuntimeMBeanException
RuntimeMXBean RuntimeOperationsException RuntimePermission SampleModel Sasl SaslClient SaslClientFactory
SaslException SaslServer SaslServerFactory Savepoint SAXParser SAXParserFactory SAXResult SAXSource
SAXTransformerFactory Scanner ScatteringByteChannel ScheduledExecutorService ScheduledFuture
ScheduledThreadPoolExecutor Schema SchemaFactory SchemaFactoryLoader SchemaViolationException Scrollable
Scrollbar ScrollBarUI ScrollPane ScrollPaneAdjustable ScrollPaneConstants ScrollPaneLayout ScrollPaneUI
SealedObject SearchControls SearchResult SecretKey SecretKeyFactory SecretKeyFactorySpi SecretKeySpec
SecureCacheResponse SecureClassLoader SecureRandom SecureRandomSpi Security SecurityException
SecurityManager SecurityPermission Segment SelectableChannel SelectionKey Selector SelectorProvider
Semaphore SeparatorUI Sequence SequenceInputStream Sequencer SerialArray SerialBlob SerialClob
SerialDatalink SerialException Serializable SerializablePermission SerialJavaObject SerialRef
SerialStruct ServerCloneException ServerError ServerException ServerNotActiveException ServerRef
ServerRuntimeException ServerSocket ServerSocketChannel ServerSocketFactory ServiceNotFoundException
ServicePermission ServiceRegistry ServiceUI ServiceUIFactory ServiceUnavailableException Set
SetOfIntegerSyntax Severity Shape ShapeGraphicAttribute SheetCollate Short ShortBuffer
ShortBufferException ShortLookupTable ShortMessage Sides Signature SignatureException SignatureSpi
SignedObject Signer SimpleAttributeSet SimpleBeanInfo SimpleDateFormat SimpleDoc SimpleFormatter
SimpleTimeZone SimpleType SinglePixelPackedSampleModel SingleSelectionModel Size2DSyntax
SizeLimitExceededException SizeRequirements SizeSequence Skeleton SkeletonMismatchException
SkeletonNotFoundException SliderUI Socket SocketAddress SocketChannel SocketException SocketFactory
SocketHandler SocketImpl SocketImplFactory SocketOptions SocketPermission SocketSecurityException
SocketTimeoutException SoftBevelBorder SoftReference SortControl SortedMap SortedSet
SortingFocusTraversalPolicy SortKey SortResponseControl Soundbank SoundbankReader SoundbankResource
Source SourceDataLine SourceLocator SpinnerDateModel SpinnerListModel SpinnerModel SpinnerNumberModel
SpinnerUI SplitPaneUI Spring SpringLayout SQLData SQLException SQLInput SQLInputImpl SQLOutput
SQLOutputImpl SQLPermission SQLWarning SSLContext SSLContextSpi SSLEngine SSLEngineResult SSLException
SSLHandshakeException SSLKeyException SSLPeerUnverifiedException SSLPermission SSLProtocolException
SslRMIClientSocketFactory SslRMIServerSocketFactory SSLServerSocket SSLServerSocketFactory SSLSession
SSLSessionBindingEvent SSLSessionBindingListener SSLSessionContext SSLSocket SSLSocketFactory Stack
StackOverflowError StackTraceElement StandardMBean StartTlsRequest StartTlsResponse StateEdit
StateEditable StateFactory Statement StreamCorruptedException StreamHandler StreamPrintService
StreamPrintServiceFactory StreamResult StreamSource StreamTokenizer StrictMath String StringBuffer
StringBufferInputStream StringBuilder StringCharacterIterator StringContent
StringIndexOutOfBoundsException StringMonitor StringMonitorMBean StringReader StringRefAddr
StringSelection StringTokenizer StringValueExp StringWriter Stroke Struct Stub StubDelegate
StubNotFoundException Style StyleConstants StyleContext StyledDocument StyledEditorKit StyleSheet
Subject SubjectDelegationPermission SubjectDomainCombiner SupportedValuesAttribute SuppressWarnings
SwingConstants SwingPropertyChangeSupport SwingUtilities SyncFactory SyncFactoryException
SyncFailedException SynchronousQueue SyncProvider SyncProviderException SyncResolver SynthConstants
SynthContext Synthesizer SynthGraphicsUtils SynthLookAndFeel SynthPainter SynthStyle SynthStyleFactory
SysexMessage System SystemColor SystemFlavorMap TabableView TabbedPaneUI TabExpander TableCellEditor
TableCellRenderer TableColumn TableColumnModel TableColumnModelEvent TableColumnModelListener
TableHeaderUI TableModel TableModelEvent TableModelListener TableUI TableView TabSet TabStop TabularData
TabularDataSupport TabularType TagElement Target TargetDataLine TargetedNotification Templates
TemplatesHandler TextAction TextArea TextAttribute TextComponent TextEvent TextField TextHitInfo
TextInputCallback TextLayout TextListener TextMeasurer TextOutputCallback TextSyntax TextUI TexturePaint
Thread ThreadDeath ThreadFactory ThreadGroup ThreadInfo ThreadLocal ThreadMXBean ThreadPoolExecutor
Throwable Tie TileObserver Time TimeLimitExceededException TimeoutException Timer
TimerAlarmClockNotification TimerMBean TimerNotification TimerTask Timestamp TimeUnit TimeZone
TitledBorder ToolBarUI Toolkit ToolTipManager ToolTipUI TooManyListenersException Track
TransactionalWriter TransactionRequiredException TransactionRolledbackException Transferable
TransferHandler TransformAttribute Transformer TransformerConfigurationException TransformerException
TransformerFactory TransformerFactoryConfigurationError TransformerHandler Transmitter Transparency
TreeCellEditor TreeCellRenderer TreeExpansionEvent TreeExpansionListener TreeMap TreeModel
TreeModelEvent TreeModelListener TreeNode TreePath TreeSelectionEvent TreeSelectionListener
TreeSelectionModel TreeSet TreeUI TreeWillExpandListener TrustAnchor TrustManager TrustManagerFactory
TrustManagerFactorySpi Type TypeInfoProvider TypeNotPresentException Types TypeVariable UID UIDefaults
UIManager UIResource UndeclaredThrowableException UndoableEdit UndoableEditEvent UndoableEditListener
UndoableEditSupport UndoManager UnexpectedException UnicastRemoteObject UnknownError
UnknownFormatConversionException UnknownFormatFlagsException UnknownGroupException UnknownHostException
UnknownObjectException UnknownServiceException UnmappableCharacterException UnmarshalException
UnmodifiableClassException UnmodifiableSetException UnrecoverableEntryException
UnrecoverableKeyException Unreferenced UnresolvedAddressException UnresolvedPermission
UnsatisfiedLinkError UnsolicitedNotification UnsolicitedNotificationEvent
UnsolicitedNotificationListener UnsupportedAddressTypeException UnsupportedAudioFileException
UnsupportedCallbackException UnsupportedCharsetException UnsupportedClassVersionError
UnsupportedEncodingException UnsupportedFlavorException UnsupportedLookAndFeelException
UnsupportedOperationException URI URIException URIResolver URISyntax URISyntaxException URL
URLClassLoader URLConnection URLDecoder URLEncoder URLStreamHandler URLStreamHandlerFactory
UTFDataFormatException Util UtilDelegate Utilities UUID Validator ValidatorHandler ValueExp ValueHandler
ValueHandlerMultiFormat VariableHeightLayoutCache Vector VerifyError VetoableChangeListener
VetoableChangeListenerProxy VetoableChangeSupport View ViewFactory ViewportLayout ViewportUI
VirtualMachineError Visibility VMID VoiceStatus Void VolatileImage WeakHashMap WeakReference WebRowSet
WildcardType Window WindowAdapter WindowConstants WindowEvent WindowFocusListener WindowListener
WindowStateListener WrappedPlainView WritableByteChannel WritableRaster WritableRenderedImage
WriteAbortedException Writer X500Principal X500PrivateCredential X509Certificate X509CertSelector
X509CRL X509CRLEntry X509CRLSelector X509EncodedKeySpec X509ExtendedKeyManager X509Extension
X509KeyManager X509TrustManager XAConnection XADataSource XAException XAResource Xid XMLConstants
XMLDecoder XMLEncoder XMLFormatter XMLGregorianCalendar XMLParseException XmlReader XmlWriter XPath
XPathConstants XPathException XPathExpression XPathExpressionException XPathFactory
XPathFactoryConfigurationException XPathFunction XPathFunctionException XPathFunctionResolver
XPathVariableResolver ZipEntry ZipException ZipFile ZipInputStream ZipOutputStream ZoneView
]
#:nocov:
end
end
end
@@ -0,0 +1,175 @@
# encoding: utf-8
module CodeRay
module Scanners
module Ruby::Patterns # :nodoc: all
KEYWORDS = %w[
and def end in or unless begin
defined? ensure module redo super until
BEGIN break do next rescue then
when END case else for retry
while alias class elsif if not return
undef yield
]
# See http://murfy.de/ruby-constants.
PREDEFINED_CONSTANTS = %w[
nil true false self
DATA ARGV ARGF ENV
FALSE TRUE NIL
STDERR STDIN STDOUT
TOPLEVEL_BINDING
RUBY_COPYRIGHT RUBY_DESCRIPTION RUBY_ENGINE RUBY_PATCHLEVEL
RUBY_PLATFORM RUBY_RELEASE_DATE RUBY_REVISION RUBY_VERSION
__FILE__ __LINE__ __ENCODING__
]
IDENT_KIND = WordList.new(:ident).
add(KEYWORDS, :keyword).
add(PREDEFINED_CONSTANTS, :predefined_constant)
KEYWORD_NEW_STATE = WordList.new(:initial).
add(%w[ def ], :def_expected).
add(%w[ undef ], :undef_expected).
add(%w[ alias ], :alias_expected).
add(%w[ class module ], :module_expected)
IDENT = 'ä'[/[[:alpha:]]/] == 'ä' ? /[[:alpha:]_][[:alnum:]_]*/ : /[^\W\d]\w*/
METHOD_NAME = / #{IDENT} [?!]? /ox
METHOD_NAME_OPERATOR = /
\*\*? # multiplication and power
| [-+~]@? # plus, minus, tilde with and without at sign
| [\/%&|^`] # division, modulo or format strings, and, or, xor, system
| \[\]=? # array getter and setter
| << | >> # append or shift left, shift right
| <=?>? | >=? # comparison, rocket operator
| ===? | =~ # simple equality, case equality, match
| ![~=@]? # negation with and without at sign, not-equal and not-match
/ox
METHOD_SUFFIX = / (?: [?!] | = (?![~>]|=(?!>)) ) /x
METHOD_NAME_EX = / #{IDENT} #{METHOD_SUFFIX}? | #{METHOD_NAME_OPERATOR} /ox
METHOD_AFTER_DOT = / #{IDENT} [?!]? | #{METHOD_NAME_OPERATOR} /ox
INSTANCE_VARIABLE = / @ #{IDENT} /ox
CLASS_VARIABLE = / @@ #{IDENT} /ox
OBJECT_VARIABLE = / @@? #{IDENT} /ox
GLOBAL_VARIABLE = / \$ (?: #{IDENT} | [1-9]\d* | 0\w* | [~&+`'=\/,;_.<>!@$?*":\\] | -[a-zA-Z_0-9] ) /ox
PREFIX_VARIABLE = / #{GLOBAL_VARIABLE} | #{OBJECT_VARIABLE} /ox
VARIABLE = / @?@? #{IDENT} | #{GLOBAL_VARIABLE} /ox
QUOTE_TO_TYPE = {
'`' => :shell,
'/'=> :regexp,
}
QUOTE_TO_TYPE.default = :string
REGEXP_MODIFIERS = /[mousenix]*/
DECIMAL = /\d+(?:_\d+)*/
OCTAL = /0_?[0-7]+(?:_[0-7]+)*/
HEXADECIMAL = /0x[0-9A-Fa-f]+(?:_[0-9A-Fa-f]+)*/
BINARY = /0b[01]+(?:_[01]+)*/
EXPONENT = / [eE] [+-]? #{DECIMAL} /ox
FLOAT_SUFFIX = / #{EXPONENT} | \. #{DECIMAL} #{EXPONENT}? /ox
FLOAT_OR_INT = / #{DECIMAL} (?: #{FLOAT_SUFFIX} () )? /ox
NUMERIC = / (?: (?=0) (?: #{OCTAL} | #{HEXADECIMAL} | #{BINARY} ) | #{FLOAT_OR_INT} ) /ox
SYMBOL = /
:
(?:
#{METHOD_NAME_EX}
| #{PREFIX_VARIABLE}
| ['"]
)
/ox
METHOD_NAME_OR_SYMBOL = / #{METHOD_NAME_EX} | #{SYMBOL} /ox
SIMPLE_ESCAPE = /
[abefnrstv]
| [0-7]{1,3}
| x[0-9A-Fa-f]{1,2}
| .
/mx
CONTROL_META_ESCAPE = /
(?: M-|C-|c )
(?: \\ (?: M-|C-|c ) )*
(?: [^\\] | \\ #{SIMPLE_ESCAPE} )?
/mox
ESCAPE = /
#{CONTROL_META_ESCAPE} | #{SIMPLE_ESCAPE}
/mox
CHARACTER = /
\?
(?:
[^\s\\]
| \\ #{ESCAPE}
)
/mox
# NOTE: This is not completely correct, but
# nobody needs heredoc delimiters ending with \n.
HEREDOC_OPEN = /
<< (-)? # $1 = float
(?:
( [A-Za-z_0-9]+ ) # $2 = delim
|
( ["'`\/] ) # $3 = quote, type
( [^\n]*? ) \3 # $4 = delim
)
/mx
RUBYDOC = /
=begin (?!\S)
.*?
(?: \Z | ^=end (?!\S) [^\n]* )
/mx
DATA = /
__END__$
.*?
(?: \Z | (?=^\#CODE) )
/mx
RUBYDOC_OR_DATA = / #{RUBYDOC} | #{DATA} /xo
# Checks for a valid value to follow. This enables
# value_expected in method calls without parentheses.
VALUE_FOLLOWS = /
(?>[ \t\f\v]+)
(?:
[%\/][^\s=]
| <<-?\S
| [-+] \d
| #{CHARACTER}
)
/ox
KEYWORDS_EXPECTING_VALUE = WordList.new.add(%w[
and end in or unless begin
defined? ensure redo super until
break do next rescue then
when case else for retry
while elsif if not return
yield
])
FANCY_STRING_START = / % ( [QqrsWwx] | (?![a-zA-Z0-9]) ) ([^a-zA-Z0-9]) /x
FANCY_STRING_KIND = Hash.new(:string).merge({
'r' => :regexp,
's' => :symbol,
'x' => :shell,
})
FANCY_STRING_INTERPRETED = Hash.new(true).merge({
'q' => false,
's' => false,
'w' => false,
})
end
end
end
@@ -0,0 +1,71 @@
# encoding: utf-8
module CodeRay
module Scanners
class Ruby
class StringState < Struct.new :type, :interpreted, :delim, :heredoc,
:opening_paren, :paren_depth, :pattern, :next_state # :nodoc: all
CLOSING_PAREN = Hash[ *%w[
( )
[ ]
< >
{ }
] ].each { |k,v| k.freeze; v.freeze } # debug, if I try to change it with <<
STRING_PATTERN = Hash.new do |h, k|
delim, interpreted = *k
# delim = delim.dup # workaround for old Ruby
delim_pattern = Regexp.escape(delim)
if closing_paren = CLOSING_PAREN[delim]
delim_pattern << Regexp.escape(closing_paren)
end
delim_pattern << '\\\\' unless delim == '\\'
# special_escapes =
# case interpreted
# when :regexp_symbols
# '| [|?*+(){}\[\].^$]'
# end
h[k] =
if interpreted && delim != '#'
/ (?= [#{delim_pattern}] | \# [{$@] ) /mx
else
/ (?= [#{delim_pattern}] ) /mx
end
end
def initialize kind, interpreted, delim, heredoc = false
if heredoc
pattern = heredoc_pattern delim, interpreted, heredoc == :indented
delim = nil
else
pattern = STRING_PATTERN[ [delim, interpreted] ]
if closing_paren = CLOSING_PAREN[delim]
opening_paren = delim
delim = closing_paren
paren_depth = 1
end
end
super kind, interpreted, delim, heredoc, opening_paren, paren_depth, pattern, :initial
end
def heredoc_pattern delim, interpreted, indented
# delim = delim.dup # workaround for old Ruby
delim_pattern = Regexp.escape(delim)
delim_pattern = / (?:\A|\n) #{ '(?>[ \t]*)' if indented } #{ Regexp.new delim_pattern } $ /x
if interpreted
/ (?= #{delim_pattern}() | \\ | \# [{$@] ) /mx # $1 set == end of heredoc
else
/ (?= #{delim_pattern}() | \\ ) /mx
end
end
end
end
end
end
+23
View File
@@ -0,0 +1,23 @@
module CodeRay
# This module holds the Style class and its subclasses.
#
# See Plugin.
module Styles
extend PluginHost
plugin_path File.dirname(__FILE__), 'styles'
# Base class for styles.
#
# Styles are used by Encoders::HTML to colorize tokens.
class Style
extend Plugin
plugin_host Styles
DEFAULT_OPTIONS = { } # :nodoc:
end
end
end
+7
View File
@@ -0,0 +1,7 @@
module CodeRay
module Styles
default :alpha
end
end
+143
View File
@@ -0,0 +1,143 @@
module CodeRay
module Styles
# A colorful theme using CSS 3 colors (with alpha channel).
class Alpha < Style
register_for :alpha
code_background = 'hsl(0,0%,95%)'
numbers_background = 'hsl(180,65%,90%)'
border_color = 'silver'
normal_color = 'black'
CSS_MAIN_STYLES = <<-MAIN # :nodoc:
.CodeRay {
background-color: #{code_background};
border: 1px solid #{border_color};
color: #{normal_color};
}
.CodeRay pre {
margin: 0px;
}
span.CodeRay { white-space: pre; border: 0px; padding: 2px; }
table.CodeRay { border-collapse: collapse; width: 100%; padding: 2px; }
table.CodeRay td { padding: 2px 4px; vertical-align: top; }
.CodeRay .line-numbers {
background-color: #{numbers_background};
color: gray;
text-align: right;
-webkit-user-select: none;
-moz-user-select: none;
user-select: none;
}
.CodeRay .line-numbers a {
background-color: #{numbers_background} !important;
color: gray !important;
text-decoration: none !important;
}
.CodeRay .line-numbers a:target { color: blue !important; }
.CodeRay .line-numbers .highlighted { color: red !important; }
.CodeRay .line-numbers .highlighted a { color: red !important; }
.CodeRay span.line-numbers { padding: 0px 4px; }
.CodeRay .line { display: block; float: left; width: 100%; }
.CodeRay .code { width: 100%; }
.CodeRay .code pre { overflow: auto; }
MAIN
TOKEN_COLORS = <<-'TOKENS'
.debug { color: white !important; background: blue !important; }
.annotation { color:#007 }
.attribute-name { color:#b48 }
.attribute-value { color:#700 }
.binary { color:#509 }
.char .content { color:#D20 }
.char .delimiter { color:#710 }
.char { color:#D20 }
.class { color:#B06; font-weight:bold }
.class-variable { color:#369 }
.color { color:#0A0 }
.comment { color:#777 }
.comment .char { color:#444 }
.comment .delimiter { color:#444 }
.complex { color:#A08 }
.constant { color:#036; font-weight:bold }
.decorator { color:#B0B }
.definition { color:#099; font-weight:bold }
.delimiter { color:black }
.directive { color:#088; font-weight:bold }
.doc { color:#970 }
.doc-string { color:#D42; font-weight:bold }
.doctype { color:#34b }
.entity { color:#800; font-weight:bold }
.error { color:#F00; background-color:#FAA }
.escape { color:#666 }
.exception { color:#C00; font-weight:bold }
.float { color:#60E }
.function { color:#06B; font-weight:bold }
.global-variable { color:#d70 }
.hex { color:#02b }
.imaginary { color:#f00 }
.include { color:#B44; font-weight:bold }
.inline { background-color: hsla(0,0%,0%,0.07); color: black }
.inline-delimiter { font-weight: bold; color: #666 }
.instance-variable { color:#33B }
.integer { color:#00D }
.key .char { color: #60f }
.key .delimiter { color: #404 }
.key { color: #606 }
.keyword { color:#080; font-weight:bold }
.label { color:#970; font-weight:bold }
.local-variable { color:#963 }
.namespace { color:#707; font-weight:bold }
.octal { color:#40E }
.operator { }
.predefined { color:#369; font-weight:bold }
.predefined-constant { color:#069 }
.predefined-type { color:#0a5; font-weight:bold }
.preprocessor { color:#579 }
.pseudo-class { color:#00C; font-weight:bold }
.regexp .content { color:#808 }
.regexp .delimiter { color:#404 }
.regexp .modifier { color:#C2C }
.regexp { background-color:hsla(300,100%,50%,0.06); }
.reserved { color:#080; font-weight:bold }
.shell .content { color:#2B2 }
.shell .delimiter { color:#161 }
.shell { background-color:hsla(120,100%,50%,0.06); }
.string .char { color: #b0b }
.string .content { color: #D20 }
.string .delimiter { color: #710 }
.string .modifier { color: #E40 }
.string { background-color:hsla(0,100%,50%,0.05); }
.symbol .content { color:#A60 }
.symbol .delimiter { color:#630 }
.symbol { color:#A60 }
.tag { color:#070 }
.type { color:#339; font-weight:bold }
.value { color: #088; }
.variable { color:#037 }
.insert { background: hsla(120,100%,50%,0.12) }
.delete { background: hsla(0,100%,50%,0.12) }
.change { color: #bbf; background: #007; }
.head { color: #f8f; background: #505 }
.head .filename { color: white; }
.delete .eyecatcher { background-color: hsla(0,100%,50%,0.2); border: 1px solid hsla(0,100%,45%,0.5); margin: -1px; border-bottom: none; border-top-left-radius: 5px; border-top-right-radius: 5px; }
.insert .eyecatcher { background-color: hsla(120,100%,50%,0.2); border: 1px solid hsla(120,100%,25%,0.5); margin: -1px; border-top: none; border-bottom-left-radius: 5px; border-bottom-right-radius: 5px; }
.insert .insert { color: #0c0; background:transparent; font-weight:bold }
.delete .delete { color: #c00; background:transparent; font-weight:bold }
.change .change { color: #88f }
.head .head { color: #f4f }
TOKENS
end
end
end
+90
View File
@@ -0,0 +1,90 @@
module CodeRay
# A Hash of all known token kinds and their associated CSS classes.
TokenKinds = Hash.new do |h, k|
warn 'Undefined Token kind: %p' % [k] if $CODERAY_DEBUG
false
end
# speedup
TokenKinds.compare_by_identity if TokenKinds.respond_to? :compare_by_identity
TokenKinds.update( # :nodoc:
:annotation => 'annotation',
:attribute_name => 'attribute-name',
:attribute_value => 'attribute-value',
:binary => 'bin',
:char => 'char',
:class => 'class',
:class_variable => 'class-variable',
:color => 'color',
:comment => 'comment',
:complex => 'complex',
:constant => 'constant',
:content => 'content',
:debug => 'debug',
:decorator => 'decorator',
:definition => 'definition',
:delimiter => 'delimiter',
:directive => 'directive',
:doc => 'doc',
:doctype => 'doctype',
:doc_string => 'doc-string',
:entity => 'entity',
:error => 'error',
:escape => 'escape',
:exception => 'exception',
:filename => 'filename',
:float => 'float',
:function => 'function',
:global_variable => 'global-variable',
:hex => 'hex',
:imaginary => 'imaginary',
:important => 'important',
:include => 'include',
:inline => 'inline',
:inline_delimiter => 'inline-delimiter',
:instance_variable => 'instance-variable',
:integer => 'integer',
:key => 'key',
:keyword => 'keyword',
:label => 'label',
:local_variable => 'local-variable',
:modifier => 'modifier',
:namespace => 'namespace',
:octal => 'octal',
:predefined => 'predefined',
:predefined_constant => 'predefined-constant',
:predefined_type => 'predefined-type',
:preprocessor => 'preprocessor',
:pseudo_class => 'pseudo-class',
:regexp => 'regexp',
:reserved => 'reserved',
:shell => 'shell',
:string => 'string',
:symbol => 'symbol',
:tag => 'tag',
:type => 'type',
:value => 'value',
:variable => 'variable',
:change => 'change',
:delete => 'delete',
:head => 'head',
:insert => 'insert',
:eyecatcher => 'eyecatcher',
:ident => false,
:operator => false,
:space => false,
:plain => false
)
TokenKinds[:method] = TokenKinds[:function]
TokenKinds[:escape] = TokenKinds[:delimiter]
TokenKinds[:docstring] = TokenKinds[:comment]
TokenKinds.freeze
end
+215
View File
@@ -0,0 +1,215 @@
module CodeRay
# GZip library for writing and reading token dumps.
autoload :GZip, 'coderay/helpers/gzip'
# = Tokens TODO: Rewrite!
#
# The Tokens class represents a list of tokens returnd from
# a Scanner.
#
# A token is not a special object, just a two-element Array
# consisting of
# * the _token_ _text_ (the original source of the token in a String) or
# a _token_ _action_ (begin_group, end_group, begin_line, end_line)
# * the _token_ _kind_ (a Symbol representing the type of the token)
#
# A token looks like this:
#
# ['# It looks like this', :comment]
# ['3.1415926', :float]
# ['$^', :error]
#
# Some scanners also yield sub-tokens, represented by special
# token actions, namely begin_group and end_group.
#
# The Ruby scanner, for example, splits "a string" into:
#
# [
# [:begin_group, :string],
# ['"', :delimiter],
# ['a string', :content],
# ['"', :delimiter],
# [:end_group, :string]
# ]
#
# Tokens is the interface between Scanners and Encoders:
# The input is split and saved into a Tokens object. The Encoder
# then builds the output from this object.
#
# Thus, the syntax below becomes clear:
#
# CodeRay.scan('price = 2.59', :ruby).html
# # the Tokens object is here -------^
#
# See how small it is? ;)
#
# Tokens gives you the power to handle pre-scanned code very easily:
# You can convert it to a webpage, a YAML file, or dump it into a gzip'ed string
# that you put in your DB.
#
# It also allows you to generate tokens directly (without using a scanner),
# to load them from a file, and still use any Encoder that CodeRay provides.
class Tokens < Array
# The Scanner instance that created the tokens.
attr_accessor :scanner
# Encode the tokens using encoder.
#
# encoder can be
# * a symbol like :html oder :statistic
# * an Encoder class
# * an Encoder object
#
# options are passed to the encoder.
def encode encoder, options = {}
encoder = Encoders[encoder].new options if encoder.respond_to? :to_sym
encoder.encode_tokens self, options
end
# Turn tokens into a string by concatenating them.
def to_s
encode CodeRay::Encoders::Encoder.new
end
# Redirects unknown methods to encoder calls.
#
# For example, if you call +tokens.html+, the HTML encoder
# is used to highlight the tokens.
def method_missing meth, options = {}
encode meth, options
rescue PluginHost::PluginNotFound
super
end
# Split the tokens into parts of the given +sizes+.
#
# The result will be an Array of Tokens objects. The parts have
# the text size specified by the parameter. In addition, each
# part closes all opened tokens. This is useful to insert tokens
# betweem them.
#
# This method is used by @Scanner#tokenize@ when called with an Array
# of source strings. The Diff encoder uses it for inline highlighting.
def split_into_parts *sizes
parts = []
opened = []
content = nil
part = Tokens.new
part_size = 0
size = sizes.first
i = 0
for item in self
case content
when nil
content = item
when String
if size && part_size + content.size > size # token must be cut
if part_size < size # some part of the token goes into this part
content = content.dup # content may no be safe to change
part << content.slice!(0, size - part_size) << item
end
# close all open groups and lines...
closing = opened.reverse.flatten.map do |content_or_kind|
case content_or_kind
when :begin_group
:end_group
when :begin_line
:end_line
else
content_or_kind
end
end
part.concat closing
begin
parts << part
part = Tokens.new
size = sizes[i += 1]
end until size.nil? || size > 0
# ...and open them again.
part.concat opened.flatten
part_size = 0
redo unless content.empty?
else
part << content << item
part_size += content.size
end
content = nil
when Symbol
case content
when :begin_group, :begin_line
opened << [content, item]
when :end_group, :end_line
opened.pop
else
raise ArgumentError, 'Unknown token action: %p, kind = %p' % [content, item]
end
part << content << item
content = nil
else
raise ArgumentError, 'Token input junk: %p, kind = %p' % [content, item]
end
end
parts << part
parts << Tokens.new while parts.size < sizes.size
parts
end
# Dumps the object into a String that can be saved
# in files or databases.
#
# The dump is created with Marshal.dump;
# In addition, it is gzipped using GZip.gzip.
#
# The returned String object includes Undumping
# so it has an #undump method. See Tokens.load.
#
# You can configure the level of compression,
# but the default value 7 should be what you want
# in most cases as it is a good compromise between
# speed and compression rate.
#
# See GZip module.
def dump gzip_level = 7
dump = Marshal.dump self
dump = GZip.gzip dump, gzip_level
dump.extend Undumping
end
# Return the actual number of tokens.
def count
size / 2
end
# Include this module to give an object an #undump
# method.
#
# The string returned by Tokens.dump includes Undumping.
module Undumping
# Calls Tokens.load with itself.
def undump
Tokens.load self
end
end
# Undump the object using Marshal.load, then
# unzip it using GZip.gunzip.
#
# The result is commonly a Tokens object, but
# this is not guaranteed.
def Tokens.load dump
dump = GZip.gunzip dump
@dump = Marshal.load dump
end
alias text_token push
def begin_group kind; push :begin_group, kind end
def end_group kind; push :end_group, kind end
def begin_line kind; push :begin_line, kind end
def end_line kind; push :end_line, kind end
alias tokens concat
end
end
+55
View File
@@ -0,0 +1,55 @@
module CodeRay
# The result of a scan operation is a TokensProxy, but should act like Tokens.
#
# This proxy makes it possible to use the classic CodeRay.scan.encode API
# while still providing the benefits of direct streaming.
class TokensProxy
attr_accessor :input, :lang, :options, :block
# Create a new TokensProxy with the arguments of CodeRay.scan.
def initialize input, lang, options = {}, block = nil
@input = input
@lang = lang
@options = options
@block = block
end
# Call CodeRay.encode if +encoder+ is a Symbol;
# otherwise, convert the receiver to tokens and call encoder.encode_tokens.
def encode encoder, options = {}
if encoder.respond_to? :to_sym
CodeRay.encode(input, lang, encoder, options)
else
encoder.encode_tokens tokens, options
end
end
# Tries to call encode;
# delegates to tokens otherwise.
def method_missing method, *args, &blk
encode method.to_sym, *args
rescue PluginHost::PluginNotFound
tokens.send(method, *args, &blk)
end
# The (cached) result of the tokenized input; a Tokens instance.
def tokens
@tokens ||= scanner.tokenize(input)
end
# A (cached) scanner instance to use for the scan task.
def scanner
@scanner ||= CodeRay.scanner(lang, options, &block)
end
# Overwrite Struct#each.
def each *args, &blk
tokens.each(*args, &blk)
self
end
end
end
+3
View File
@@ -0,0 +1,3 @@
module CodeRay
VERSION = '1.0.0'
end
+304
View File
@@ -0,0 +1,304 @@
# encoding: utf-8
require 'test/unit'
require File.expand_path('../../lib/assert_warning', __FILE__)
$:.unshift File.expand_path('../../../lib', __FILE__)
require 'coderay'
class BasicTest < Test::Unit::TestCase
def test_version
assert_nothing_raised do
assert_match(/\A\d\.\d\.\d?\z/, CodeRay::VERSION)
end
end
RUBY_TEST_CODE = 'puts "Hello, World!"'
RUBY_TEST_TOKENS = [
['puts', :ident],
[' ', :space],
[:begin_group, :string],
['"', :delimiter],
['Hello, World!', :content],
['"', :delimiter],
[:end_group, :string]
].flatten
def test_simple_scan
assert_nothing_raised do
assert_equal RUBY_TEST_TOKENS, CodeRay.scan(RUBY_TEST_CODE, :ruby).tokens
end
end
RUBY_TEST_HTML = 'puts <span class="string"><span class="delimiter">&quot;</span>' +
'<span class="content">Hello, World!</span><span class="delimiter">&quot;</span></span>'
def test_simple_highlight
assert_nothing_raised do
assert_equal RUBY_TEST_HTML, CodeRay.scan(RUBY_TEST_CODE, :ruby).html
end
end
def test_scan_file
CodeRay.scan_file __FILE__
end
def test_encode
assert_equal 1, CodeRay.encode('test', :python, :count)
end
def test_encode_tokens
assert_equal 1, CodeRay.encode_tokens(CodeRay::Tokens['test', :string], :count)
end
def test_encode_file
assert_equal File.read(__FILE__), CodeRay.encode_file(__FILE__, :text)
end
def test_highlight
assert_match '<pre>test</pre>', CodeRay.highlight('test', :python)
end
def test_highlight_file
assert_match "require <span class=\"string\"><span class=\"delimiter\">'</span><span class=\"content\">test/unit</span><span class=\"delimiter\">'</span></span>\n", CodeRay.highlight_file(__FILE__)
end
def test_duo
assert_equal(RUBY_TEST_CODE,
CodeRay::Duo[:plain, :text].highlight(RUBY_TEST_CODE))
assert_equal(RUBY_TEST_CODE,
CodeRay::Duo[:plain => :text].highlight(RUBY_TEST_CODE))
end
def test_duo_stream
assert_equal(RUBY_TEST_CODE,
CodeRay::Duo[:plain, :text].highlight(RUBY_TEST_CODE, :stream => true))
end
def test_comment_filter
assert_equal <<-EXPECTED, CodeRay.scan(<<-INPUT, :ruby).comment_filter.text
#!/usr/bin/env ruby
code
more code
EXPECTED
#!/usr/bin/env ruby
=begin
A multi-line comment.
=end
code
# A single-line comment.
more code # and another comment, in-line.
INPUT
end
def test_lines_of_code
assert_equal 2, CodeRay.scan(<<-INPUT, :ruby).lines_of_code
#!/usr/bin/env ruby
=begin
A multi-line comment.
=end
code
# A single-line comment.
more code # and another comment, in-line.
INPUT
rHTML = <<-RHTML
<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Strict//EN"
"http://www.w3.org/TR/xhtml1/DTD/xhtml1-strict.dtd">
<html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en" lang="en">
<head>
<meta http-equiv="content-type" content="text/html;charset=UTF-8" />
<title><%= controller.controller_name.titleize %>: <%= controller.action_name %></title>
<%= stylesheet_link_tag 'scaffold' %>
</head>
<body>
<p style="color: green"><%= flash[:notice] %></p>
<div id="main">
<%= yield %>
</div>
</body>
</html>
RHTML
assert_equal 0, CodeRay.scan(rHTML, :html).lines_of_code
assert_equal 0, CodeRay.scan(rHTML, :php).lines_of_code
assert_equal 0, CodeRay.scan(rHTML, :yaml).lines_of_code
assert_equal 4, CodeRay.scan(rHTML, :erb).lines_of_code
end
def test_list_of_encoders
assert_kind_of(Array, CodeRay::Encoders.list)
assert CodeRay::Encoders.list.include?(:count)
end
def test_list_of_scanners
assert_kind_of(Array, CodeRay::Scanners.list)
assert CodeRay::Scanners.list.include?(:text)
end
def test_token_kinds
assert_kind_of Hash, CodeRay::TokenKinds
for kind, css_class in CodeRay::TokenKinds
assert_kind_of Symbol, kind
if css_class != false
assert_kind_of String, css_class, "TokenKinds[%p] == %p" % [kind, css_class]
end
end
assert_equal 'reserved', CodeRay::TokenKinds[:reserved]
assert_warning 'Undefined Token kind: :shibboleet' do
assert_equal false, CodeRay::TokenKinds[:shibboleet]
end
end
class Milk < CodeRay::Encoders::Encoder
FILE_EXTENSION = 'cocoa'
end
class HoneyBee < CodeRay::Encoders::Encoder
end
def test_encoder_file_extension
assert_nothing_raised do
assert_equal 'html', CodeRay::Encoders::Page::FILE_EXTENSION
assert_equal 'cocoa', Milk::FILE_EXTENSION
assert_equal 'cocoa', Milk.new.file_extension
assert_equal 'honeybee', HoneyBee::FILE_EXTENSION
assert_equal 'honeybee', HoneyBee.new.file_extension
end
assert_raise NameError do
HoneyBee::MISSING_CONSTANT
end
end
def test_encoder_tokens
encoder = CodeRay::Encoders::Encoder.new
encoder.send :setup, {}
assert_raise(ArgumentError) { encoder.token :strange, '' }
encoder.token 'test', :debug
end
def test_encoder_deprecated_interface
encoder = CodeRay::Encoders::Encoder.new
encoder.send :setup, {}
assert_warning 'Using old Tokens#<< interface.' do
encoder << ['test', :content]
end
assert_raise ArgumentError do
encoder << [:strange, :input]
end
assert_raise ArgumentError do
encoder.encode_tokens [['test', :token]]
end
end
def encoder_token_interface_deprecation_warning_given
CodeRay::Encoders::Encoder.send :class_variable_get, :@@CODERAY_TOKEN_INTERFACE_DEPRECATION_WARNING_GIVEN
end
def test_scanner_file_extension
assert_equal 'rb', CodeRay::Scanners::Ruby.file_extension
assert_equal 'rb', CodeRay::Scanners::Ruby.new.file_extension
assert_equal 'java', CodeRay::Scanners::Java.file_extension
assert_equal 'java', CodeRay::Scanners::Java.new.file_extension
end
def test_scanner_lang
assert_equal :ruby, CodeRay::Scanners::Ruby.lang
assert_equal :ruby, CodeRay::Scanners::Ruby.new.lang
assert_equal :java, CodeRay::Scanners::Java.lang
assert_equal :java, CodeRay::Scanners::Java.new.lang
end
def test_scanner_tokenize
assert_equal ['foo', :plain], CodeRay::Scanners::Plain.new.tokenize('foo')
assert_equal [['foo', :plain], ['bar', :plain]], CodeRay::Scanners::Plain.new.tokenize(['foo', 'bar'])
CodeRay::Scanners::Plain.new.tokenize 42
end
def test_scanner_tokens
scanner = CodeRay::Scanners::Plain.new
scanner.tokenize('foo')
assert_equal ['foo', :plain], scanner.tokens
scanner.string = ''
assert_equal ['', :plain], scanner.tokens
end
def test_scanner_line_and_column
scanner = CodeRay::Scanners::Plain.new "foo\nbär+quux"
assert_equal 0, scanner.pos
assert_equal 1, scanner.line
assert_equal 1, scanner.column
scanner.scan(/foo/)
assert_equal 3, scanner.pos
assert_equal 1, scanner.line
assert_equal 4, scanner.column
scanner.scan(/\n/)
assert_equal 4, scanner.pos
assert_equal 2, scanner.line
assert_equal 1, scanner.column
scanner.scan(/b/)
assert_equal 5, scanner.pos
assert_equal 2, scanner.line
assert_equal 2, scanner.column
scanner.scan(/a/)
assert_equal 5, scanner.pos
assert_equal 2, scanner.line
assert_equal 2, scanner.column
scanner.scan(/ä/)
assert_equal 7, scanner.pos
assert_equal 2, scanner.line
assert_equal 4, scanner.column
scanner.scan(/r/)
assert_equal 8, scanner.pos
assert_equal 2, scanner.line
assert_equal 5, scanner.column
end
def test_scanner_use_subclasses
assert_raise NotImplementedError do
CodeRay::Scanners::Scanner.new
end
end
class InvalidScanner < CodeRay::Scanners::Scanner
end
def test_scanner_scan_tokens
assert_raise NotImplementedError do
InvalidScanner.new.tokenize ''
end
end
class RaisingScanner < CodeRay::Scanners::Scanner
def scan_tokens encoder, options
raise_inspect 'message', [], :initial
end
end
def test_scanner_raise_inspect
assert_raise CodeRay::Scanners::Scanner::ScanError do
RaisingScanner.new.tokenize ''
end
end
def test_scan_a_frozen_string
assert_nothing_raised do
CodeRay.scan RUBY_VERSION, :ruby
CodeRay.scan RUBY_VERSION, :plain
end
end
def test_scan_a_non_string
assert_nothing_raised do
CodeRay.scan 42, :ruby
CodeRay.scan nil, :ruby
CodeRay.scan self, :ruby
CodeRay.encode ENV.to_hash, :ruby, :page
CodeRay.highlight CodeRay, :plain
end
end
end
+130
View File
@@ -0,0 +1,130 @@
require 'test/unit'
$:.unshift File.expand_path('../../../lib', __FILE__)
require 'coderay'
class ExamplesTest < Test::Unit::TestCase
def test_examples
# output as HTML div (using inline CSS styles)
div = CodeRay.scan('puts "Hello, world!"', :ruby).div
assert_equal <<-DIV, div
<div class="CodeRay">
<div class="code"><pre>puts <span style="background-color:hsla(0,100%,50%,0.05)"><span style="color:#710">&quot;</span><span style="color:#D20">Hello, world!</span><span style="color:#710">&quot;</span></span></pre></div>
</div>
DIV
# ...with line numbers
div = CodeRay.scan(<<-CODE.chomp, :ruby).div(:line_numbers => :table)
5.times do
puts 'Hello, world!'
end
CODE
assert_equal <<-DIV, div
<table class="CodeRay"><tr>
<td class="line-numbers" title="double click to toggle" ondblclick="with (this.firstChild.style) { display = (display == '') ? 'none' : '' }"><pre><a href="#n1" name="n1">1</a>
<a href="#n2" name="n2">2</a>
<a href="#n3" name="n3">3</a>
</pre></td>
<td class="code"><pre><span style="color:#00D">5</span>.times <span style="color:#080;font-weight:bold">do</span>
puts <span style="background-color:hsla(0,100%,50%,0.05)"><span style="color:#710">'</span><span style="color:#D20">Hello, world!</span><span style="color:#710">'</span></span>
<span style="color:#080;font-weight:bold">end</span></pre></td>
</tr></table>
DIV
# output as standalone HTML page (using CSS classes)
page = CodeRay.scan('puts "Hello, world!"', :ruby).page
assert page[<<-PAGE]
<body style="background-color: white;">
<table class="CodeRay"><tr>
<td class="line-numbers" title="double click to toggle" ondblclick="with (this.firstChild.style) { display = (display == '') ? 'none' : '' }"><pre>
</pre></td>
<td class="code"><pre>puts <span class="string"><span class="delimiter">&quot;</span><span class="content">Hello, world!</span><span class="delimiter">&quot;</span></span></pre></td>
</tr></table>
</body>
PAGE
# keep scanned tokens for later use
tokens = CodeRay.scan('{ "just": "an", "example": 42 }', :json)
assert_kind_of CodeRay::TokensProxy, tokens
assert_equal ["{", :operator, " ", :space, :begin_group, :key,
"\"", :delimiter, "just", :content, "\"", :delimiter,
:end_group, :key, ":", :operator, " ", :space,
:begin_group, :string, "\"", :delimiter, "an", :content,
"\"", :delimiter, :end_group, :string, ",", :operator,
" ", :space, :begin_group, :key, "\"", :delimiter,
"example", :content, "\"", :delimiter, :end_group, :key,
":", :operator, " ", :space, "42", :integer,
" ", :space, "}", :operator], tokens.tokens
# produce a token statistic
assert_equal <<-STATISTIC, tokens.statistic
Code Statistics
Tokens 26
Non-Whitespace 15
Bytes Total 31
Token Types (7):
type count ratio size (average)
-------------------------------------------------------------
TOTAL 26 100.00 % 1.2
delimiter 6 23.08 % 1.0
operator 5 19.23 % 1.0
space 5 19.23 % 1.0
key 4 15.38 % 0.0
:begin_group 3 11.54 % 0.0
:end_group 3 11.54 % 0.0
content 3 11.54 % 4.3
string 2 7.69 % 0.0
integer 1 3.85 % 2.0
STATISTIC
# count the tokens
assert_equal 26, tokens.count
# produce a HTML div, but with CSS classes
div = tokens.div(:css => :class)
assert_equal <<-DIV, div
<div class="CodeRay">
<div class="code"><pre>{ <span class="key"><span class="delimiter">&quot;</span><span class="content">just</span><span class="delimiter">&quot;</span></span>: <span class="string"><span class="delimiter">&quot;</span><span class="content">an</span><span class="delimiter">&quot;</span></span>, <span class="key"><span class="delimiter">&quot;</span><span class="content">example</span><span class="delimiter">&quot;</span></span>: <span class="integer">42</span> }</pre></div>
</div>
DIV
# highlight a file (HTML div); guess the file type base on the extension
require 'coderay/helpers/file_type'
assert_equal :ruby, CodeRay::FileType[__FILE__]
# get a new scanner for Python
python_scanner = CodeRay.scanner :python
assert_kind_of CodeRay::Scanners::Python, python_scanner
# get a new encoder for terminal
terminal_encoder = CodeRay.encoder :term
assert_kind_of CodeRay::Encoders::Terminal, terminal_encoder
# scanning into tokens
tokens = python_scanner.tokenize 'import this; # The Zen of Python'
assert_equal ["import", :keyword, " ", :space, "this", :include,
";", :operator, " ", :space, "# The Zen of Python", :comment], tokens
# format the tokens
term = terminal_encoder.encode_tokens(tokens)
assert_equal "\e[1;31mimport\e[0m \e[33mthis\e[0m; \e[37m# The Zen of Python\e[0m", term
# re-using scanner and encoder
ruby_highlighter = CodeRay::Duo[:ruby, :div]
div = ruby_highlighter.encode('puts "Hello, world!"')
assert_equal <<-DIV, div
<div class="CodeRay">
<div class="code"><pre>puts <span style="background-color:hsla(0,100%,50%,0.05)"><span style="color:#710">&quot;</span><span style="color:#D20">Hello, world!</span><span style="color:#710">&quot;</span></span></pre></div>
</div>
DIV
end
end
@@ -0,0 +1,84 @@
require 'test/unit'
require File.expand_path('../../lib/assert_warning', __FILE__)
$:.unshift File.expand_path('../../../lib', __FILE__)
require 'coderay'
begin
require 'rubygems' unless defined? Gem
gem 'RedCloth', '>= 4.0.3' rescue nil
require 'redcloth'
rescue LoadError
warn 'RedCloth not found - skipping for_redcloth tests.'
undef RedCloth if defined? RedCloth
end
class BasicTest < Test::Unit::TestCase
def test_for_redcloth
require 'coderay/for_redcloth'
assert_equal "<p><span lang=\"ruby\" class=\"CodeRay\">puts <span style=\"background-color:hsla(0,100%,50%,0.05)\"><span style=\"color:#710\">&quot;</span><span style=\"color:#D20\">Hello, World!</span><span style=\"color:#710\">&quot;</span></span></span></p>",
RedCloth.new('@[ruby]puts "Hello, World!"@').to_html
assert_equal <<-BLOCKCODE.chomp,
<div lang="ruby" class="CodeRay">
<div class="code"><pre>puts <span style="background-color:hsla(0,100%,50%,0.05)"><span style="color:#710">&quot;</span><span style="color:#D20">Hello, World!</span><span style="color:#710">&quot;</span></span></pre></div>
</div>
BLOCKCODE
RedCloth.new('bc[ruby]. puts "Hello, World!"').to_html
end
def test_for_redcloth_no_lang
require 'coderay/for_redcloth'
assert_equal "<p><code>puts \"Hello, World!\"</code></p>",
RedCloth.new('@puts "Hello, World!"@').to_html
assert_equal <<-BLOCKCODE.chomp,
<pre><code>puts \"Hello, World!\"</code></pre>
BLOCKCODE
RedCloth.new('bc. puts "Hello, World!"').to_html
end
def test_for_redcloth_style
require 'coderay/for_redcloth'
assert_equal <<-BLOCKCODE.chomp,
<pre style=\"color: red;\"><code style=\"color: red;\">puts \"Hello, World!\"</code></pre>
BLOCKCODE
RedCloth.new('bc{color: red}. puts "Hello, World!"').to_html
end
def test_for_redcloth_escapes
require 'coderay/for_redcloth'
assert_equal '<p><span lang="ruby" class="CodeRay">&gt;</span></p>',
RedCloth.new('@[ruby]>@').to_html
assert_equal <<-BLOCKCODE.chomp,
<div lang="ruby" class="CodeRay">
<div class="code"><pre>&amp;</pre></div>
</div>
BLOCKCODE
RedCloth.new('bc[ruby]. &').to_html
end
def test_for_redcloth_escapes2
require 'coderay/for_redcloth'
assert_equal "<p><span lang=\"c\" class=\"CodeRay\"><span style=\"color:#579\">#include</span> <span style=\"color:#B44;font-weight:bold\">&lt;test.h&gt;</span></span></p>",
RedCloth.new('@[c]#include <test.h>@').to_html
end
# See http://jgarber.lighthouseapp.com/projects/13054/tickets/124-code-markup-does-not-allow-brackets.
def test_for_redcloth_false_positive
require 'coderay/for_redcloth'
assert_warning 'CodeRay::Scanners could not load plugin :project; falling back to :text' do
assert_equal '<p><code>[project]_dff.skjd</code></p>',
RedCloth.new('@[project]_dff.skjd@').to_html
end
# false positive, but expected behavior / known issue
assert_equal "<p><span lang=\"ruby\" class=\"CodeRay\">_dff.skjd</span></p>",
RedCloth.new('@[ruby]_dff.skjd@').to_html
assert_warning 'CodeRay::Scanners could not load plugin :project; falling back to :text' do
assert_equal <<-BLOCKCODE.chomp,
<pre><code>[project]_dff.skjd</code></pre>
BLOCKCODE
RedCloth.new('bc. [project]_dff.skjd').to_html
end
end
end if defined? RedCloth
+15
View File
@@ -0,0 +1,15 @@
require 'test/unit'
$VERBOSE = $CODERAY_DEBUG = true
$:.unshift File.expand_path('../../../lib', __FILE__)
require 'coderay'
mydir = File.dirname(__FILE__)
suite = Dir[File.join(mydir, '*.rb')].
map { |tc| File.basename(tc).sub(/\.rb$/, '') } - %w'suite for_redcloth'
puts "Running basic CodeRay #{CodeRay::VERSION} tests: #{suite.join(', ')}"
for test_case in suite
load File.join(mydir, test_case + '.rb')
end